2023-10-15T16:04:25,959 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x2ae1c993 to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:25,965 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4718bc37, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:25,965 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:25,976 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.wal-reader.8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.1697385854712 to pos 0, reset compression=false 2023-10-15T16:04:25,981 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f510014 connected 2023-10-15T16:04:25,989 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.wal-reader.8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,40585,1697385851811/8c5d72df62de%2C40585%2C1697385851811.1697385854701 to pos 0, reset compression=false 2023-10-15T16:04:25,995 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699 to pos 349, reset compression=false 2023-10-15T16:04:26,006 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,45999,1697385859285 (queues=1) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:26,006 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C45999%2C1697385859285 2023-10-15T16:04:26,006 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,45999,1697385859285, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:26,006 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:26,006 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C45999%2C1697385859285 2023-10-15T16:04:26,006 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385861037, startPosition=0, beingWritten=true 2023-10-15T16:04:26,007 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@280979f6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:26,008 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x6243043c to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:26,025 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,38435,1697385859175 (queues=1) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:26,025 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C38435%2C1697385859175 2023-10-15T16:04:26,025 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,38435,1697385859175, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:26,027 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385861137, startPosition=0, beingWritten=true 2023-10-15T16:04:26,027 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:26,036 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f510016 connected 2023-10-15T16:04:26,036 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C38435%2C1697385859175 2023-10-15T16:04:26,038 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@ffd39d2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:26,039 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceWALReader(175): Read 0 WAL entries eligible for replication 2023-10-15T16:04:26,039 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385861037 to pos 349, reset compression=false 2023-10-15T16:04:26,039 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,45999,1697385859285 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385861037, lastWalPosition=349, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-10-15T16:04:26,051 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699, startPosition=349, beingWritten=true 2023-10-15T16:04:26,053 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385861037, startPosition=349, beingWritten=true 2023-10-15T16:04:26,055 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,46527,1697385859222 (queues=1) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:26,055 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C46527%2C1697385859222 2023-10-15T16:04:26,055 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,46527,1697385859222, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:26,061 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385861048, startPosition=0, beingWritten=true 2023-10-15T16:04:26,061 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.wal-reader.8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.1697385854712, startPosition=0, beingWritten=true 2023-10-15T16:04:26,065 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C38435%2C1697385859175]: currently replicating from: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385861137 at position: -1 2023-10-15T16:04:26,065 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.wal-reader.8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,40585,1697385851811/8c5d72df62de%2C40585%2C1697385851811.1697385854701, startPosition=0, beingWritten=true 2023-10-15T16:04:26,077 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C46527%2C1697385859222 2023-10-15T16:04:26,112 INFO [Listener at localhost/35085 {}] hbase.ResourceChecker(147): before: replication.TestSyncReplicationStandbyKillRS#testStandbyKillRegionServer Thread=755, OpenFileDescriptor=1192, MaxFileDescriptor=1048576, SystemLoadAverage=865, ProcessCount=14, AvailableMemoryMB=8608 2023-10-15T16:04:26,112 WARN [Listener at localhost/35085 {}] hbase.ResourceChecker(130): Thread=755 is superior to 500 2023-10-15T16:04:26,112 WARN [Listener at localhost/35085 {}] hbase.ResourceChecker(130): OpenFileDescriptor=1192 is superior to 1024 2023-10-15T16:04:26,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.HMaster(3935): Client=jenkins//172.17.0.3 transit current cluster state to STANDBY in a synchronous replication peer id=1 2023-10-15T16:04:26,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] procedure2.ProcedureExecutor(1032): Stored pid=14, state=RUNNABLE:PRE_PEER_SYNC_REPLICATION_STATE_TRANSITION, hasLock=false; org.apache.hadoop.hbase.master.replication.TransitPeerSyncReplicationStateProcedure 2023-10-15T16:04:26,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=14 2023-10-15T16:04:26,143 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=16, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-10-15T16:04:26,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=14 2023-10-15T16:04:26,265 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385861137 to pos 0, reset compression=false 2023-10-15T16:04:26,285 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385861137, startPosition=0, beingWritten=true 2023-10-15T16:04:26,299 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38435 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=15 2023-10-15T16:04:26,299 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45999 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-10-15T16:04:26,299 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46527 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-10-15T16:04:26,299 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:26,299 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:26,299 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:26,300 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] regionserver.ReplicationSourceManager(490): Terminate replication source for 1 2023-10-15T16:04:26,300 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,46527,1697385859222 because: Peer 1 state or config changed. Will close the previous replication source and open a new one 2023-10-15T16:04:26,300 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] regionserver.ReplicationSourceManager(490): Terminate replication source for 1 2023-10-15T16:04:26,300 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,38435,1697385859175 because: Peer 1 state or config changed. Will close the previous replication source and open a new one 2023-10-15T16:04:26,302 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] regionserver.ReplicationSourceManager(490): Terminate replication source for 1 2023-10-15T16:04:26,302 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,45999,1697385859285 because: Peer 1 state or config changed. Will close the previous replication source and open a new one 2023-10-15T16:04:26,408 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f510016, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:26,408 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] client.AsyncConnectionImpl(233): Connection has been closed by RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0. 2023-10-15T16:04:26,409 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.refreshSources(ReplicationSourceManager.java:492) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.transitSyncReplicationPeerState(PeerProcedureHandlerImpl.java:176) at org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable.doCall(RefreshPeerCallable.java:66) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:35) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:23) at org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:51) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:26,410 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:26,410 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x6243043c to 127.0.0.1:52753 2023-10-15T16:04:26,412 DEBUG [HBase-Metrics2-1 {}] regionserver.MetricsTableSourceImpl(133): Creating new MetricsTableSourceImpl for table 'hbase:replication' 2023-10-15T16:04:26,417 DEBUG [HBase-Metrics2-1 {}] regionserver.MetricsTableSourceImpl(133): Creating new MetricsTableSourceImpl for table 'SyncRep' 2023-10-15T16:04:26,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=14 2023-10-15T16:04:26,508 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f510012, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:26,508 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] client.AsyncConnectionImpl(233): Connection has been closed by RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0. 2023-10-15T16:04:26,508 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.refreshSources(ReplicationSourceManager.java:492) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.transitSyncReplicationPeerState(PeerProcedureHandlerImpl.java:176) at org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable.doCall(RefreshPeerCallable.java:66) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:35) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:23) at org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:51) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:26,508 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:26,508 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x392cb9b2 to 127.0.0.1:52753 2023-10-15T16:04:26,511 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) ~[?:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:307) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) ~[classes/:?] 2023-10-15T16:04:26,601 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.wal-reader.8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,40585,1697385851811/8c5d72df62de%2C40585%2C1697385851811.1697385854701 to pos 0, reset compression=false 2023-10-15T16:04:26,604 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.wal-reader.8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.1697385854712 to pos 0, reset compression=false 2023-10-15T16:04:26,608 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699 to pos 349, reset compression=false 2023-10-15T16:04:26,608 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f510014, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:26,608 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] client.AsyncConnectionImpl(233): Connection has been closed by RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0. 2023-10-15T16:04:26,608 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.refreshSources(ReplicationSourceManager.java:492) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.transitSyncReplicationPeerState(PeerProcedureHandlerImpl.java:176) at org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable.doCall(RefreshPeerCallable.java:66) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:35) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:23) at org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:51) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:26,608 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:26,609 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x2ae1c993 to 127.0.0.1:52753 2023-10-15T16:04:26,609 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) ~[?:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:307) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) ~[classes/:?] 2023-10-15T16:04:26,609 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.wal-reader.8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,40585,1697385851811/8c5d72df62de%2C40585%2C1697385851811.1697385854701, startPosition=0, beingWritten=true 2023-10-15T16:04:26,611 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 terminated 2023-10-15T16:04:26,617 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.wal-reader.8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.1697385854712, startPosition=0, beingWritten=true 2023-10-15T16:04:26,618 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699, startPosition=349, beingWritten=true 2023-10-15T16:04:26,640 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,46527,1697385859222, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:26,640 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] regionserver.ReplicationSourceManager(500): Startup replication source for 1 2023-10-15T16:04:26,655 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:26,658 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x47771406 to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:26,660 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:26,664 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f510018 connected 2023-10-15T16:04:26,668 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@f65f7bf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:26,709 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 terminated 2023-10-15T16:04:26,709 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) ~[?:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:307) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) ~[classes/:?] 2023-10-15T16:04:26,712 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=17 2023-10-15T16:04:26,712 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,46527,1697385859222 (queues=1) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:26,713 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C46527%2C1697385859222 2023-10-15T16:04:26,713 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,46527,1697385859222, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:26,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=17 2023-10-15T16:04:26,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,46527,1697385859222 suceeded 2023-10-15T16:04:26,714 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C46527%2C1697385859222 2023-10-15T16:04:26,717 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1413): Finished pid=17, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 572 msec 2023-10-15T16:04:26,727 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,45999,1697385859285, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:26,727 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] regionserver.ReplicationSourceManager(500): Startup replication source for 1 2023-10-15T16:04:26,728 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=16 2023-10-15T16:04:26,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=16 2023-10-15T16:04:26,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,45999,1697385859285 suceeded 2023-10-15T16:04:26,733 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=16, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 588 msec 2023-10-15T16:04:26,743 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:26,746 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x1178fc78 to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:26,748 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:26,754 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f51001a connected 2023-10-15T16:04:26,755 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@5cf8fc11, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:26,757 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,45999,1697385859285 (queues=1) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:26,757 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C45999%2C1697385859285 2023-10-15T16:04:26,757 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,45999,1697385859285, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:26,758 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C45999%2C1697385859285 2023-10-15T16:04:26,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=14 2023-10-15T16:04:26,809 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 terminated 2023-10-15T16:04:26,834 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,38435,1697385859175, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:26,834 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] regionserver.ReplicationSourceManager(500): Startup replication source for 1 2023-10-15T16:04:26,849 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=15 2023-10-15T16:04:26,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=15 2023-10-15T16:04:26,850 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,38435,1697385859175 suceeded 2023-10-15T16:04:26,860 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=15, resume processing ppid=14 2023-10-15T16:04:26,860 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=15, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 711 msec 2023-10-15T16:04:26,870 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:26,916 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x7f813361 to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:26,916 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:26,918 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=18, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=19, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=20, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-10-15T16:04:26,944 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f51001c connected 2023-10-15T16:04:26,948 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@4d23161, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:26,950 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,38435,1697385859175 (queues=1) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:26,950 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C38435%2C1697385859175 2023-10-15T16:04:26,950 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,38435,1697385859175, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:26,952 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C38435%2C1697385859175 2023-10-15T16:04:27,004 WARN [HBase-Metrics2-1 {}] impl.MetricsConfig(136): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-10-15T16:04:27,073 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46527 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=20 2023-10-15T16:04:27,073 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45999 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=19 2023-10-15T16:04:27,073 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38435 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=18 2023-10-15T16:04:27,073 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:27,073 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:27,078 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:27,092 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,46527,1697385859222, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:27,095 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,45999,1697385859285, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:27,098 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,38435,1697385859175, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:27,115 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,45999,1697385859285, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:27,115 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] regionserver.ReplicationSourceManager(431): Terminate replication source for 1 2023-10-15T16:04:27,115 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,45999,1697385859285 because: Sync replication peer 1 is transiting to STANDBY. Will close the previous replication source and open a new one 2023-10-15T16:04:27,120 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,46527,1697385859222, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:27,120 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,38435,1697385859175, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:27,120 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] regionserver.ReplicationSourceManager(431): Terminate replication source for 1 2023-10-15T16:04:27,120 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] regionserver.ReplicationSourceManager(431): Terminate replication source for 1 2023-10-15T16:04:27,120 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,46527,1697385859222 because: Sync replication peer 1 is transiting to STANDBY. Will close the previous replication source and open a new one 2023-10-15T16:04:27,120 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,38435,1697385859175 because: Sync replication peer 1 is transiting to STANDBY. Will close the previous replication source and open a new one 2023-10-15T16:04:27,218 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f51001a, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:27,219 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] client.AsyncConnectionImpl(233): Connection has been closed by RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0. 2023-10-15T16:04:27,219 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:664) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:659) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.drainSources(ReplicationSourceManager.java:432) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.transitSyncReplicationPeerState(PeerProcedureHandlerImpl.java:197) at org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable.doCall(RefreshPeerCallable.java:66) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:35) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:23) at org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:51) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:27,219 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:27,219 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x1178fc78 to 127.0.0.1:52753 2023-10-15T16:04:27,219 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 terminated 2023-10-15T16:04:27,223 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] regionserver.ReplicationSourceManager(439): Startup replication source for 1 2023-10-15T16:04:27,224 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(198): WAL AsyncFSWAL 8c5d72df62de%2C45999%2C1697385859285:(num 1697385861037) roll requested 2023-10-15T16:04:27,226 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699 to pos 349, reset compression=false 2023-10-15T16:04:27,226 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.wal-reader.8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,40585,1697385851811/8c5d72df62de%2C40585%2C1697385851811.1697385854701 to pos 0, reset compression=false 2023-10-15T16:04:27,227 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.wal-reader.8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.1697385854712 to pos 0, reset compression=false 2023-10-15T16:04:27,240 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, exclude list is [], retry=0 2023-10-15T16:04:27,244 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:27,245 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699, startPosition=349, beingWritten=true 2023-10-15T16:04:27,245 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:27,246 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:27,246 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:27,246 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.wal-reader.8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,40585,1697385851811/8c5d72df62de%2C40585%2C1697385851811.1697385854701, startPosition=0, beingWritten=true 2023-10-15T16:04:27,258 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:27,258 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x4871b63d to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:27,261 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.wal-reader.8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.1697385854712, startPosition=0, beingWritten=true 2023-10-15T16:04:27,277 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f51001e connected 2023-10-15T16:04:27,278 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(976): Rolled WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385861037 with entries=1, filesize=349 B; new WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 2023-10-15T16:04:27,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=14 2023-10-15T16:04:27,282 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK], DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK], DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK]] 2023-10-15T16:04:27,282 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(868): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385861037 is not closed yet, will try archiving it next time 2023-10-15T16:04:27,283 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(198): WAL AsyncFSWAL 8c5d72df62de%2C45999%2C1697385859285.meta:.meta(num 1697385861367) roll requested 2023-10-15T16:04:27,287 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@27cafb2c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:27,290 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,45999,1697385859285 (queues=1) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:27,290 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C45999%2C1697385859285 2023-10-15T16:04:27,290 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,45999,1697385859285, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:27,293 INFO [WAL-Archive-0 {}] wal.AbstractFSWAL(957): Archiving hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385861037 to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C45999%2C1697385859285.1697385861037 2023-10-15T16:04:27,293 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C45999%2C1697385859285 2023-10-15T16:04:27,302 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta, exclude list is [], retry=0 2023-10-15T16:04:27,305 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:27,305 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:27,305 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:27,307 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(976): Rolled WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta with entries=13, filesize=3.43 KB; new WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta 2023-10-15T16:04:27,308 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK], DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK], DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK]] 2023-10-15T16:04:27,308 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(868): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta is not closed yet, will try archiving it next time 2023-10-15T16:04:27,318 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f510018, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:27,318 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] client.AsyncConnectionImpl(233): Connection has been closed by RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0. 2023-10-15T16:04:27,319 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:664) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:659) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.drainSources(ReplicationSourceManager.java:432) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.transitSyncReplicationPeerState(PeerProcedureHandlerImpl.java:197) at org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable.doCall(RefreshPeerCallable.java:66) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:35) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:23) at org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:51) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:27,319 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:27,319 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x47771406 to 127.0.0.1:52753 2023-10-15T16:04:27,319 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 terminated 2023-10-15T16:04:27,322 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] regionserver.ReplicationSourceManager(439): Startup replication source for 1 2023-10-15T16:04:27,322 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(198): WAL AsyncFSWAL 8c5d72df62de%2C46527%2C1697385859222:(num 1697385861048) roll requested 2023-10-15T16:04:27,324 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=19}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=19 2023-10-15T16:04:27,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=19 2023-10-15T16:04:27,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,45999,1697385859285 suceeded 2023-10-15T16:04:27,330 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=19, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 409 msec 2023-10-15T16:04:27,341 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, exclude list is [], retry=0 2023-10-15T16:04:27,343 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:27,344 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:27,344 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:27,344 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:27,350 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x2106402e to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:27,356 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:27,359 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(976): Rolled WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385861048 with entries=0, filesize=85 B; new WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 2023-10-15T16:04:27,363 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f510020 connected 2023-10-15T16:04:27,363 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK], DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK], DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK]] 2023-10-15T16:04:27,363 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(868): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385861048 is not closed yet, will try archiving it next time 2023-10-15T16:04:27,367 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@9176274, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:27,370 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,46527,1697385859222 (queues=1) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:27,371 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C46527%2C1697385859222 2023-10-15T16:04:27,371 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,46527,1697385859222, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:27,371 INFO [WAL-Archive-0 {}] wal.AbstractFSWAL(957): Archiving hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385861048 to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C46527%2C1697385859222.1697385861048 2023-10-15T16:04:27,381 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C46527%2C1697385859222 2023-10-15T16:04:27,418 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f51001c, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:27,419 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] client.AsyncConnectionImpl(233): Connection has been closed by RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0. 2023-10-15T16:04:27,420 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:664) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:659) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.drainSources(ReplicationSourceManager.java:432) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.transitSyncReplicationPeerState(PeerProcedureHandlerImpl.java:197) at org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable.doCall(RefreshPeerCallable.java:66) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:35) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:23) at org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:51) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:27,420 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:27,420 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x7f813361 to 127.0.0.1:52753 2023-10-15T16:04:27,420 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 terminated 2023-10-15T16:04:27,423 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=20}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=20 2023-10-15T16:04:27,423 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] regionserver.ReplicationSourceManager(439): Startup replication source for 1 2023-10-15T16:04:27,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=20 2023-10-15T16:04:27,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,46527,1697385859222 suceeded 2023-10-15T16:04:27,425 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(198): WAL AsyncFSWAL 8c5d72df62de%2C38435%2C1697385859175.rep:(num 1697385865402) roll requested 2023-10-15T16:04:27,429 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=20, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 508 msec 2023-10-15T16:04:27,440 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:27,443 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x2ffdd842 to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:27,445 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.rep.1697385867425, exclude list is [], retry=0 2023-10-15T16:04:27,445 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:27,449 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f510022 connected 2023-10-15T16:04:27,457 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:27,457 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:27,457 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:27,457 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@3c93ae71, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:27,463 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,38435,1697385859175 (queues=0) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:27,464 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(976): Rolled WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.rep.1697385865402 with entries=11, filesize=2.64 KB; new WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.rep.1697385867425 2023-10-15T16:04:27,465 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK], DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK], DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK]] 2023-10-15T16:04:27,465 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(868): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.rep.1697385865402 is not closed yet, will try archiving it next time 2023-10-15T16:04:27,465 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(198): WAL AsyncFSWAL 8c5d72df62de%2C38435%2C1697385859175:(num 1697385861137) roll requested 2023-10-15T16:04:27,479 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465, exclude list is [], retry=0 2023-10-15T16:04:27,483 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:27,483 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:27,483 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:27,486 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(976): Rolled WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385861137 with entries=0, filesize=85 B; new WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465 2023-10-15T16:04:27,487 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C38435%2C1697385859175 2023-10-15T16:04:27,487 INFO [regionserver/8c5d72df62de:0.logRoller {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,38435,1697385859175, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:27,493 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK], DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK], DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK]] 2023-10-15T16:04:27,493 INFO [regionserver/8c5d72df62de:0.logRoller.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C38435%2C1697385859175 2023-10-15T16:04:27,493 DEBUG [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractFSWAL(868): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385861137 is not closed yet, will try archiving it next time 2023-10-15T16:04:27,494 INFO [WAL-Archive-0 {}] wal.AbstractFSWAL(957): Archiving hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385861137 to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.1697385861137 2023-10-15T16:04:27,524 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=18}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=18 2023-10-15T16:04:27,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=18 2023-10-15T16:04:27,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,38435,1697385859175 suceeded 2023-10-15T16:04:27,532 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=18, resume processing ppid=14 2023-10-15T16:04:27,532 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1413): Finished pid=18, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 610 msec 2023-10-15T16:04:27,536 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=21, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=22, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=23, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-10-15T16:04:27,691 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46527 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=23 2023-10-15T16:04:27,691 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45999 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=22 2023-10-15T16:04:27,691 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38435 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=21 2023-10-15T16:04:27,691 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=23}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=ENABLE_PEER 2023-10-15T16:04:27,691 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=22}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=ENABLE_PEER 2023-10-15T16:04:27,691 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=21}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=ENABLE_PEER 2023-10-15T16:04:27,692 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=21}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=21 2023-10-15T16:04:27,692 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=22}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=22 2023-10-15T16:04:27,692 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=23}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=23 2023-10-15T16:04:27,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=22 2023-10-15T16:04:27,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for ENABLE on 8c5d72df62de,45999,1697385859285 suceeded 2023-10-15T16:04:27,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=23 2023-10-15T16:04:27,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=21 2023-10-15T16:04:27,693 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for ENABLE on 8c5d72df62de,46527,1697385859222 suceeded 2023-10-15T16:04:27,693 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for ENABLE on 8c5d72df62de,38435,1697385859175 suceeded 2023-10-15T16:04:27,699 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:27,705 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=22, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 165 msec 2023-10-15T16:04:27,707 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=21, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 168 msec 2023-10-15T16:04:27,709 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=23, resume processing ppid=14 2023-10-15T16:04:27,709 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1413): Finished pid=23, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 170 msec 2023-10-15T16:04:27,711 INFO [PEWorker-5 {}] replication.TransitPeerSyncReplicationStateProcedure(135): Successfully transit current cluster state from DOWNGRADE_ACTIVE to STANDBY for sync replication peer 1 2023-10-15T16:04:27,714 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1413): Finished pid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.TransitPeerSyncReplicationStateProcedure in 1.5910 sec 2023-10-15T16:04:27,781 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:27,789 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=true 2023-10-15T16:04:27,907 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 0, reset compression=false 2023-10-15T16:04:27,913 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:27,979 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699 to pos 349, reset compression=false 2023-10-15T16:04:27,979 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.wal-reader.8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,40585,1697385851811/8c5d72df62de%2C40585%2C1697385851811.1697385854701 to pos 0, reset compression=false 2023-10-15T16:04:27,984 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699, startPosition=349, beingWritten=true 2023-10-15T16:04:27,984 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.wal-reader.8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,40585,1697385851811/8c5d72df62de%2C40585%2C1697385851811.1697385854701, startPosition=0, beingWritten=true 2023-10-15T16:04:27,984 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.wal-reader.8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.1697385854712 to pos 0, reset compression=false 2023-10-15T16:04:27,987 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:27,992 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.wal-reader.8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.1697385854712, startPosition=0, beingWritten=true 2023-10-15T16:04:27,993 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:27,994 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465 to pos 0, reset compression=false 2023-10-15T16:04:27,999 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=true 2023-10-15T16:04:28,219 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 0, reset compression=false 2023-10-15T16:04:28,225 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:28,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=14 2023-10-15T16:04:28,291 INFO [RPCClient-NioEventLoopGroup-7-8 {}] client.RawAsyncHBaseAdmin$ReplicationProcedureBiConsumer(2911): Operation: TRANSIT_REPLICATION_PEER_SYNCHRONOUS_REPLICATION_STATE, peerId: 1 completed 2023-10-15T16:04:28,293 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41825 {}] master.HMaster(3935): Client=jenkins//172.17.0.3 transit current cluster state to ACTIVE in a synchronous replication peer id=1 2023-10-15T16:04:28,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41825 {}] procedure2.ProcedureExecutor(1032): Stored pid=14, state=RUNNABLE:PRE_PEER_SYNC_REPLICATION_STATE_TRANSITION, hasLock=false; org.apache.hadoop.hbase.master.replication.TransitPeerSyncReplicationStateProcedure 2023-10-15T16:04:28,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41825 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=14 2023-10-15T16:04:28,298 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:28,301 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=16, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=17, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-10-15T16:04:28,306 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:28,306 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465 to pos 0, reset compression=false 2023-10-15T16:04:28,315 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=true 2023-10-15T16:04:28,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41825 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=14 2023-10-15T16:04:28,456 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41815 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=17 2023-10-15T16:04:28,456 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:28,456 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34443 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=15 2023-10-15T16:04:28,456 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40585 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=16 2023-10-15T16:04:28,456 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:28,456 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:28,457 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=17}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=17 2023-10-15T16:04:28,457 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=15}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=15 2023-10-15T16:04:28,458 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=16}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=16 2023-10-15T16:04:28,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41825 {}] master.HMaster(4067): Remote procedure done, pid=17 2023-10-15T16:04:28,458 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41825 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,41815,1697385851922 suceeded 2023-10-15T16:04:28,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41825 {}] master.HMaster(4067): Remote procedure done, pid=15 2023-10-15T16:04:28,459 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41825 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,34443,1697385851561 suceeded 2023-10-15T16:04:28,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.HMaster(4067): Remote procedure done, pid=16 2023-10-15T16:04:28,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,40585,1697385851811 suceeded 2023-10-15T16:04:28,464 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1413): Finished pid=15, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 160 msec 2023-10-15T16:04:28,464 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=17, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 160 msec 2023-10-15T16:04:28,465 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=16, resume processing ppid=14 2023-10-15T16:04:28,465 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=16, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 160 msec 2023-10-15T16:04:28,467 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=18, ppid=14, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS, hasLock=false; ReopenTableRegionsProcedure table=SyncRep}] 2023-10-15T16:04:28,475 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE, hasLock=false; TransitRegionStateProcedure table=SyncRep, region=9352e165439825b87cc19da7b1c32db3, REOPEN/MOVE}] 2023-10-15T16:04:28,476 INFO [PEWorker-5 {}] procedure.MasterProcedureScheduler(792): Took xlock for pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE, hasLock=false; TransitRegionStateProcedure table=SyncRep, region=9352e165439825b87cc19da7b1c32db3, REOPEN/MOVE 2023-10-15T16:04:28,480 INFO [PEWorker-5 {}] assignment.RegionStateStore(223): pid=19 updating hbase:meta row=9352e165439825b87cc19da7b1c32db3, regionState=CLOSING, regionLocation=8c5d72df62de,41815,1697385851922 2023-10-15T16:04:28,483 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE, hasLock=false; CloseRegionProcedure 9352e165439825b87cc19da7b1c32db3, server=8c5d72df62de,41815,1697385851922}] 2023-10-15T16:04:28,532 INFO [RS:2;8c5d72df62de:41815 {}] wal.AbstractFSWAL(602): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=8c5d72df62de%2C41815%2C1697385851922-1697385868530-1, suffix=.syncrep, logDir=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922, archiveDir=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/oldWALs, maxLogs=10 2023-10-15T16:04:28,546 DEBUG [RS:2;8c5d72df62de:41815 {}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep, exclude list is [], retry=0 2023-10-15T16:04:28,549 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-6-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:39187,DS-eaaea501-b641-4fca-a984-24cc07de73c3,DISK] 2023-10-15T16:04:28,549 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-6-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34495,DS-fc348dad-bfbe-4fbb-9f7c-bb4c9671de8d,DISK] 2023-10-15T16:04:28,549 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-6-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33713,DS-fa8dc875-9f1a-4495-9419-6dc5471ce174,DISK] 2023-10-15T16:04:28,578 DEBUG [RS:2;8c5d72df62de:41815 {}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/remoteWALs/1/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep, exclude list is [], retry=0 2023-10-15T16:04:28,586 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-6-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:28,586 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-6-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:28,586 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-6-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:28,595 INFO [RS:2;8c5d72df62de:41815 {}] wal.AbstractFSWAL(980): New WAL /user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep 2023-10-15T16:04:28,596 DEBUG [RS:2;8c5d72df62de:41815 {}] regionserver.ReplicationSourceManager(789): Start tracking logs for wal group 8c5d72df62de%2C41815%2C1697385851922-1697385868530-1 for peer 1 2023-10-15T16:04:28,596 DEBUG [RS:2;8c5d72df62de:41815 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C41815%2C1697385851922-1697385868530-1 2023-10-15T16:04:28,596 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,41815,1697385851922, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:28,602 DEBUG [RS:2;8c5d72df62de:41815 {}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [] 2023-10-15T16:04:28,602 INFO [RS:2;8c5d72df62de:41815.replicationSource.shipper8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C41815%2C1697385851922-1697385868530-1 2023-10-15T16:04:28,602 DEBUG [RS:2;8c5d72df62de:41815.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep, startPosition=0, beingWritten=true 2023-10-15T16:04:28,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=14 2023-10-15T16:04:28,629 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 0, reset compression=false 2023-10-15T16:04:28,636 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:28,644 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION, pid=20}] handler.UnassignRegionHandler(117): Close 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,645 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION, pid=20}] regionserver.HRegion(1710): Closing 9352e165439825b87cc19da7b1c32db3, disabling compactions & flushes 2023-10-15T16:04:28,646 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION, pid=20}] regionserver.HRegion(1743): Closing region SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:28,646 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION, pid=20}] regionserver.HRegion(1764): Time limited wait for close lock on SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:28,646 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION, pid=20}] regionserver.HRegion(1831): Acquired close lock on SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. after waiting 0 ms 2023-10-15T16:04:28,646 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION, pid=20}] regionserver.HRegion(1841): Updates disabled for region SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:28,654 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION, pid=20}] wal.WALSplitUtil(410): Wrote file=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/default/SyncRep/9352e165439825b87cc19da7b1c32db3/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-10-15T16:04:28,656 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION, pid=20}] regionserver.HRegion(1961): Closed SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:28,656 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION, pid=20}] regionserver.HRegion(1664): Region close journal for 9352e165439825b87cc19da7b1c32db3: Waiting for close lock at 1697385868644Running coprocessor pre-close hooks at 1697385868644Disabling compacts and flushes for region at 1697385868644Disabling writes for close at 1697385868646 (+2 ms)Writing region close event to WAL at 1697385868647 (+1 ms)Running coprocessor post-close hooks at 1697385868655 (+8 ms)Closed at 1697385868656 (+1 ms) 2023-10-15T16:04:28,656 WARN [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION, pid=20}] regionserver.HRegionServer(3243): Not adding moved region record: 9352e165439825b87cc19da7b1c32db3 to self. 2023-10-15T16:04:28,660 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION, pid=20}] handler.UnassignRegionHandler(150): Closed 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,661 INFO [PEWorker-4 {}] assignment.RegionStateStore(223): pid=19 updating hbase:meta row=9352e165439825b87cc19da7b1c32db3, regionState=CLOSED 2023-10-15T16:04:28,666 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=20, resume processing ppid=19 2023-10-15T16:04:28,666 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=20, ppid=19, state=SUCCESS, hasLock=false; CloseRegionProcedure 9352e165439825b87cc19da7b1c32db3, server=8c5d72df62de,41815,1697385851922 in 180 msec 2023-10-15T16:04:28,667 INFO [PEWorker-2 {}] assignment.TransitRegionStateProcedure(262): Starting pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true; TransitRegionStateProcedure table=SyncRep, region=9352e165439825b87cc19da7b1c32db3, REOPEN/MOVE; state=CLOSED, location=8c5d72df62de,41815,1697385851922; forceNewPlan=false, retain=true 2023-10-15T16:04:28,715 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:28,720 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465 to pos 0, reset compression=false 2023-10-15T16:04:28,721 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:28,726 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=true 2023-10-15T16:04:28,789 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699 to pos 349, reset compression=false 2023-10-15T16:04:28,792 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.wal-reader.8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,40585,1697385851811/8c5d72df62de%2C40585%2C1697385851811.1697385854701 to pos 0, reset compression=false 2023-10-15T16:04:28,793 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699, startPosition=349, beingWritten=true 2023-10-15T16:04:28,797 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.wal-reader.8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,40585,1697385851811/8c5d72df62de%2C40585%2C1697385851811.1697385854701, startPosition=0, beingWritten=true 2023-10-15T16:04:28,799 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.wal-reader.8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.1697385854712 to pos 0, reset compression=false 2023-10-15T16:04:28,799 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSourceWALReader(175): Read 0 WAL entries eligible for replication 2023-10-15T16:04:28,800 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699 to pos 614, reset compression=false 2023-10-15T16:04:28,799 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.shipper8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,41815,1697385851922 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699, lastWalPosition=614, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-10-15T16:04:28,805 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.wal-reader.8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.1697385854712, startPosition=0, beingWritten=true 2023-10-15T16:04:28,808 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699, startPosition=614, beingWritten=true 2023-10-15T16:04:28,810 DEBUG [RS:2;8c5d72df62de:41815.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep to pos 0, reset compression=false 2023-10-15T16:04:28,815 DEBUG [RS:2;8c5d72df62de:41815.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep, startPosition=0, beingWritten=true 2023-10-15T16:04:28,817 INFO [8c5d72df62de:41825 {}] balancer.BaseLoadBalancer(387): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-10-15T16:04:28,818 INFO [PEWorker-1 {}] assignment.RegionStateStore(223): pid=19 updating hbase:meta row=9352e165439825b87cc19da7b1c32db3, regionState=OPENING, regionLocation=8c5d72df62de,41815,1697385851922 2023-10-15T16:04:28,821 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=21, ppid=19, state=RUNNABLE, hasLock=false; OpenRegionProcedure 9352e165439825b87cc19da7b1c32db3, server=8c5d72df62de,41815,1697385851922}] 2023-10-15T16:04:28,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=14 2023-10-15T16:04:28,980 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] handler.AssignRegionHandler(127): Open SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:28,980 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(7727): Opening region: {ENCODED => 9352e165439825b87cc19da7b1c32db3, NAME => 'SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3.', STARTKEY => '', ENDKEY => ''} 2023-10-15T16:04:28,981 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table SyncRep 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,981 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(891): Instantiated SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=0 ; minColumnNum=100 ; preparePutThreadLimit=0 ; hotProtect now disable 2023-10-15T16:04:28,981 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(7769): checking encryption for 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,981 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(7772): checking classloading for 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,983 INFO [StoreOpener-9352e165439825b87cc19da7b1c32db3-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,984 INFO [StoreOpener-9352e165439825b87cc19da7b1c32db3-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9352e165439825b87cc19da7b1c32db3 columnFamilyName cf 2023-10-15T16:04:28,984 DEBUG [StoreOpener-9352e165439825b87cc19da7b1c32db3-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:28,985 INFO [StoreOpener-9352e165439825b87cc19da7b1c32db3-1 {}] regionserver.HStore(324): Store=9352e165439825b87cc19da7b1c32db3/cf, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:28,985 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(1028): replaying wal for 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,986 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/default/SyncRep/9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,988 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/default/SyncRep/9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,989 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(1038): stopping wal replay for 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,989 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(1050): Cleaning up temporary data for 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,991 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(1083): writing seq id for 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,996 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(1104): Opened 9352e165439825b87cc19da7b1c32db3; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65700458, jitterRate=-0.020986884832382202}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-15T16:04:28,996 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(1112): Running coprocessor post-open hooks for 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:28,996 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegion(996): Region open journal for 9352e165439825b87cc19da7b1c32db3: Running coprocessor pre-open hook at 1697385868981Writing region info on filesystem at 1697385868981Initializing all the Stores at 1697385868982 (+1 ms)Instantiating store for column family {NAME => 'cf', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385868982Cleaning up temporary data from old regions at 1697385868989 (+7 ms)Running coprocessor post-open hooks at 1697385868996 (+7 ms)Region opened successfully at 1697385868996 2023-10-15T16:04:29,000 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegionServer(2186): Post open deploy tasks for SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3., pid=21, masterSystemTime=1697385868974 2023-10-15T16:04:29,004 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] regionserver.HRegionServer(2216): Finished post open deploy task for SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:29,004 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=21}] handler.AssignRegionHandler(147): Opened SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:29,005 INFO [PEWorker-3 {}] assignment.RegionStateStore(223): pid=19 updating hbase:meta row=9352e165439825b87cc19da7b1c32db3, regionState=OPEN, repBarrier=5, openSeqNum=5, regionLocation=8c5d72df62de,41815,1697385851922 2023-10-15T16:04:29,012 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=21, resume processing ppid=19 2023-10-15T16:04:29,012 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=21, ppid=19, state=SUCCESS, hasLock=false; OpenRegionProcedure 9352e165439825b87cc19da7b1c32db3, server=8c5d72df62de,41815,1697385851922 in 187 msec 2023-10-15T16:04:29,014 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699 to pos 614, reset compression=false 2023-10-15T16:04:29,019 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=19, resume processing ppid=18 2023-10-15T16:04:29,021 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=19, ppid=18, state=SUCCESS, hasLock=false; TransitRegionStateProcedure table=SyncRep, region=9352e165439825b87cc19da7b1c32db3, REOPEN/MOVE in 537 msec 2023-10-15T16:04:29,023 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699, startPosition=614, beingWritten=true 2023-10-15T16:04:29,027 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=18, resume processing ppid=14 2023-10-15T16:04:29,028 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=18, ppid=14, state=SUCCESS, hasLock=false; ReopenTableRegionsProcedure table=SyncRep in 557 msec 2023-10-15T16:04:29,041 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=22, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=23, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=24, ppid=14, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-10-15T16:04:29,120 DEBUG [RS:2;8c5d72df62de:41815.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep to pos 0, reset compression=false 2023-10-15T16:04:29,125 DEBUG [RS:2;8c5d72df62de:41815.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep, startPosition=0, beingWritten=true 2023-10-15T16:04:29,130 DEBUG [RS:2;8c5d72df62de:41815.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSourceWALReader(175): Read 0 WAL entries eligible for replication 2023-10-15T16:04:29,130 DEBUG [RS:2;8c5d72df62de:41815.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep to pos 349, reset compression=false 2023-10-15T16:04:29,130 DEBUG [RS:2;8c5d72df62de:41815.replicationSource.shipper8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,41815,1697385851922 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep, lastWalPosition=349, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-10-15T16:04:29,134 DEBUG [RS:2;8c5d72df62de:41815.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep, startPosition=349, beingWritten=true 2023-10-15T16:04:29,142 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 0, reset compression=false 2023-10-15T16:04:29,148 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:29,196 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34443 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=22 2023-10-15T16:04:29,196 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=40585 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=23 2023-10-15T16:04:29,196 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=23}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:29,196 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=22}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:29,196 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=22}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=22 2023-10-15T16:04:29,196 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=23}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=23 2023-10-15T16:04:29,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.HMaster(4067): Remote procedure done, pid=22 2023-10-15T16:04:29,197 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41825 {}] master.HMaster(4067): Remote procedure done, pid=23 2023-10-15T16:04:29,197 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41815 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=24 2023-10-15T16:04:29,197 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41825 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,40585,1697385851811 suceeded 2023-10-15T16:04:29,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,34443,1697385851561 suceeded 2023-10-15T16:04:29,197 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=24}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:29,198 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=24}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=24 2023-10-15T16:04:29,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.HMaster(4067): Remote procedure done, pid=24 2023-10-15T16:04:29,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,41815,1697385851922 suceeded 2023-10-15T16:04:29,203 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=22, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 168 msec 2023-10-15T16:04:29,203 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1413): Finished pid=23, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 168 msec 2023-10-15T16:04:29,205 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=24, resume processing ppid=14 2023-10-15T16:04:29,205 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1413): Finished pid=24, ppid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 160 msec 2023-10-15T16:04:29,206 INFO [PEWorker-5 {}] replication.TransitPeerSyncReplicationStateProcedure(135): Successfully transit current cluster state from DOWNGRADE_ACTIVE to ACTIVE for sync replication peer 1 2023-10-15T16:04:29,209 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1413): Finished pid=14, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.TransitPeerSyncReplicationStateProcedure in 913 msec 2023-10-15T16:04:29,228 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:29,230 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465 to pos 0, reset compression=false 2023-10-15T16:04:29,235 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:29,237 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=true 2023-10-15T16:04:29,332 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699 to pos 614, reset compression=false 2023-10-15T16:04:29,337 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699, startPosition=614, beingWritten=true 2023-10-15T16:04:29,339 DEBUG [RS:2;8c5d72df62de:41815.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep to pos 349, reset compression=false 2023-10-15T16:04:29,344 DEBUG [RS:2;8c5d72df62de:41815.replicationSource.wal-reader.8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep, startPosition=349, beingWritten=true 2023-10-15T16:04:29,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=14 2023-10-15T16:04:29,430 INFO [RPCClient-NioEventLoopGroup-7-7 {}] client.RawAsyncHBaseAdmin$ReplicationProcedureBiConsumer(2911): Operation: TRANSIT_REPLICATION_PEER_SYNCHRONOUS_REPLICATION_STATE, peerId: 1 completed 2023-10-15T16:04:29,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.HMaster(3890): Client=jenkins//172.17.0.3 disable replication peer, id=1 2023-10-15T16:04:29,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] procedure2.ProcedureExecutor(1032): Stored pid=25, state=RUNNABLE:PRE_PEER_MODIFICATION, hasLock=false; org.apache.hadoop.hbase.master.replication.DisablePeerProcedure 2023-10-15T16:04:29,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=25 2023-10-15T16:04:29,443 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=27, ppid=25, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=28, ppid=25, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-10-15T16:04:29,544 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C40585%2C1697385851811]: currently replicating from: hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,40585,1697385851811/8c5d72df62de%2C40585%2C1697385851811.1697385854701 at position: -1 2023-10-15T16:04:29,549 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C41815%2C1697385851922]: currently replicating from: hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922.1697385854699 at position: 614 walGroup [8c5d72df62de%2C41815%2C1697385851922-1697385868530-1]: currently replicating from: hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,41815,1697385851922/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep at position: 349 2023-10-15T16:04:29,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=25 2023-10-15T16:04:29,553 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C34443%2C1697385851561]: currently replicating from: hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.1697385854712 at position: -1 2023-10-15T16:04:29,596 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41815 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=28 2023-10-15T16:04:29,596 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34443 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=26 2023-10-15T16:04:29,596 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=40585 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=27 2023-10-15T16:04:29,597 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=DISABLE_PEER 2023-10-15T16:04:29,597 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=DISABLE_PEER 2023-10-15T16:04:29,597 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=DISABLE_PEER 2023-10-15T16:04:29,597 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] regionserver.ReplicationSourceManager(490): Terminate replication source for 1 2023-10-15T16:04:29,597 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] regionserver.ReplicationSourceManager(490): Terminate replication source for 1 2023-10-15T16:04:29,597 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] regionserver.ReplicationSourceManager(490): Terminate replication source for 1 2023-10-15T16:04:29,597 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,34443,1697385851561 because: Peer 1 state or config changed. Will close the previous replication source and open a new one 2023-10-15T16:04:29,597 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,40585,1697385851811 because: Peer 1 state or config changed. Will close the previous replication source and open a new one 2023-10-15T16:04:29,597 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,41815,1697385851922 because: Peer 1 state or config changed. Will close the previous replication source and open a new one 2023-10-15T16:04:29,701 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f51000d, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:29,701 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] client.AsyncConnectionImpl(233): Connection has been closed by RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0. 2023-10-15T16:04:29,701 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.refreshSources(ReplicationSourceManager.java:492) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.refreshPeerState(PeerProcedureHandlerImpl.java:89) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.disablePeer(PeerProcedureHandlerImpl.java:108) at org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable.doCall(RefreshPeerCallable.java:60) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:35) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:23) at org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:51) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:29,701 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:29,701 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x0d7f11bd to 127.0.0.1:52753 2023-10-15T16:04:29,753 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 0, reset compression=false 2023-10-15T16:04:29,758 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:29,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=25 2023-10-15T16:04:29,801 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f51000c, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:29,801 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] client.AsyncConnectionImpl(233): Connection has been closed by RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0. 2023-10-15T16:04:29,801 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.refreshSources(ReplicationSourceManager.java:492) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.refreshPeerState(PeerProcedureHandlerImpl.java:89) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.disablePeer(PeerProcedureHandlerImpl.java:108) at org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable.doCall(RefreshPeerCallable.java:60) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:35) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:23) at org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:51) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:29,801 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:29,801 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x56207b4e to 127.0.0.1:52753 2023-10-15T16:04:29,801 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.shipper8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) ~[?:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:307) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) ~[classes/:?] 2023-10-15T16:04:29,840 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:29,842 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465 to pos 0, reset compression=false 2023-10-15T16:04:29,845 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:29,847 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=true 2023-10-15T16:04:29,901 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f51000b, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:29,901 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] client.AsyncConnectionImpl(233): Connection has been closed by RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0. 2023-10-15T16:04:29,901 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.refreshSources(ReplicationSourceManager.java:492) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.refreshPeerState(PeerProcedureHandlerImpl.java:89) at org.apache.hadoop.hbase.replication.regionserver.PeerProcedureHandlerImpl.disablePeer(PeerProcedureHandlerImpl.java:108) at org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable.doCall(RefreshPeerCallable.java:60) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:35) at org.apache.hadoop.hbase.procedure2.BaseRSProcedureCallable.call(BaseRSProcedureCallable.java:23) at org.apache.hadoop.hbase.regionserver.handler.RSProcedureHandler.process(RSProcedureHandler.java:51) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:29,901 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:29,902 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.shipper8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 terminated 2023-10-15T16:04:29,902 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x1e961cca to 127.0.0.1:52753 2023-10-15T16:04:29,902 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.shipper8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) ~[?:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:307) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) ~[classes/:?] 2023-10-15T16:04:29,917 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,34443,1697385851561, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:29,918 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] regionserver.ReplicationSourceManager(500): Startup replication source for 1 2023-10-15T16:04:29,918 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=26 2023-10-15T16:04:29,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.HMaster(4067): Remote procedure done, pid=26 2023-10-15T16:04:29,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for DISABLE on 8c5d72df62de,34443,1697385851561 suceeded 2023-10-15T16:04:29,923 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1413): Finished pid=26, ppid=25, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 478 msec 2023-10-15T16:04:29,932 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:29,935 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x24556a9b to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:29,937 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:29,938 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f510024 connected 2023-10-15T16:04:29,939 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@2e56e00d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:29,946 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,34443,1697385851561 (queues=1) is replicating from cluster=241aca37-6ead-421a-9a95-62a3de5ca646 to cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:04:29,946 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C34443%2C1697385851561 2023-10-15T16:04:29,946 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,34443,1697385851561, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:29,947 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.shipper8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C34443%2C1697385851561 2023-10-15T16:04:30,002 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.shipper8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 terminated 2023-10-15T16:04:30,002 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.shipper8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) ~[?:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:307) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) ~[classes/:?] 2023-10-15T16:04:30,018 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,40585,1697385851811, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:30,018 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] regionserver.ReplicationSourceManager(500): Startup replication source for 1 2023-10-15T16:04:30,019 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=27 2023-10-15T16:04:30,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.HMaster(4067): Remote procedure done, pid=27 2023-10-15T16:04:30,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for DISABLE on 8c5d72df62de,40585,1697385851811 suceeded 2023-10-15T16:04:30,023 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=27, ppid=25, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 578 msec 2023-10-15T16:04:30,035 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:30,042 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x1fc01338 to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:30,043 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:30,044 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f510026 connected 2023-10-15T16:04:30,045 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@60f87e65, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:30,046 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,40585,1697385851811 (queues=1) is replicating from cluster=241aca37-6ead-421a-9a95-62a3de5ca646 to cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:04:30,046 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C40585%2C1697385851811 2023-10-15T16:04:30,046 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,40585,1697385851811, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:30,047 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.shipper8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C40585%2C1697385851811 2023-10-15T16:04:30,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=25 2023-10-15T16:04:30,202 WARN [RS:2;8c5d72df62de:41815.replicationSource.shipper8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) ~[?:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:307) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) ~[classes/:?] 2023-10-15T16:04:30,302 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.shipper8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 terminated 2023-10-15T16:04:30,302 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS:2;8c5d72df62de:41815.replicationSource.shipper8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 terminated 2023-10-15T16:04:30,317 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,41815,1697385851922, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:30,318 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] regionserver.ReplicationSourceManager(500): Startup replication source for 1 2023-10-15T16:04:30,318 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=28}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=28 2023-10-15T16:04:30,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.HMaster(4067): Remote procedure done, pid=28 2023-10-15T16:04:30,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for DISABLE on 8c5d72df62de,41815,1697385851922 suceeded 2023-10-15T16:04:30,324 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=28, resume processing ppid=25 2023-10-15T16:04:30,324 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=28, ppid=25, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 878 msec 2023-10-15T16:04:30,324 INFO [PEWorker-4 {}] replication.DisablePeerProcedure(67): Successfully disabled peer 1 2023-10-15T16:04:30,326 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=25, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.DisablePeerProcedure in 890 msec 2023-10-15T16:04:30,337 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:30,339 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x4910ae8c to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:30,341 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:30,342 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f510028 connected 2023-10-15T16:04:30,343 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@75bd882f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:30,344 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,41815,1697385851922 (queues=2) is replicating from cluster=241aca37-6ead-421a-9a95-62a3de5ca646 to cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:04:30,344 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C41815%2C1697385851922 2023-10-15T16:04:30,344 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,41815,1697385851922, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:30,344 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C41815%2C1697385851922-1697385868530-1 2023-10-15T16:04:30,344 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.shipper8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C41815%2C1697385851922 2023-10-15T16:04:30,344 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,41815,1697385851922, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:30,353 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.shipper8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C41815%2C1697385851922-1697385868530-1 2023-10-15T16:04:30,463 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 0, reset compression=false 2023-10-15T16:04:30,468 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:30,549 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:30,552 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465 to pos 0, reset compression=false 2023-10-15T16:04:30,555 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:30,557 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=true 2023-10-15T16:04:30,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41825 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=25 2023-10-15T16:04:30,581 INFO [RPCClient-NioEventLoopGroup-7-7 {}] client.RawAsyncHBaseAdmin$ReplicationProcedureBiConsumer(2911): Operation: DISABLE_REPLICATION_PEER, peerId: 1 completed 2023-10-15T16:04:30,584 DEBUG [RPCClient-NioEventLoopGroup-7-3 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'SyncRep', row='\x00\x00\x00\x00', locateType=CURRENT is [region=SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3., hostname=8c5d72df62de,41815,1697385851922, seqNum=5] 2023-10-15T16:04:30,900 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C45999%2C1697385859285]: currently replicating from: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 at position: -1 2023-10-15T16:04:30,918 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C46527%2C1697385859222]: currently replicating from: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 at position: -1 2023-10-15T16:04:31,011 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C38435%2C1697385859175]: currently replicating from: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465 at position: -1 2023-10-15T16:04:31,086 DEBUG [HBase-Metrics2-1 {}] impl.GlobalMetricRegistriesAdapter(131): Registering adapter for the MetricRegistry: RegionServer,sub=TableRequests_Namespace_hbase_table_replication 2023-10-15T16:04:31,087 INFO [HBase-Metrics2-1 {}] impl.GlobalMetricRegistriesAdapter(135): Registering RegionServer,sub=TableRequests_Namespace_hbase_table_replication Metrics about Tables on a single HBase RegionServer 2023-10-15T16:04:31,088 DEBUG [HBase-Metrics2-1 {}] impl.GlobalMetricRegistriesAdapter(131): Registering adapter for the MetricRegistry: RegionServer,sub=TableRequests_Namespace_default_table_SyncRep 2023-10-15T16:04:31,088 INFO [HBase-Metrics2-1 {}] impl.GlobalMetricRegistriesAdapter(135): Registering RegionServer,sub=TableRequests_Namespace_default_table_SyncRep Metrics about Tables on a single HBase RegionServer 2023-10-15T16:04:31,272 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 0, reset compression=false 2023-10-15T16:04:31,282 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:31,359 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:31,362 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465 to pos 0, reset compression=false 2023-10-15T16:04:31,366 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:31,368 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=true 2023-10-15T16:04:32,190 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 0, reset compression=false 2023-10-15T16:04:32,199 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:32,273 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:32,274 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465 to pos 0, reset compression=false 2023-10-15T16:04:32,279 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:32,285 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=true 2023-10-15T16:04:32,549 INFO [Listener at localhost/35085 {}] hbase.HBaseTestingUtil(1041): Shutting down minicluster 2023-10-15T16:04:32,549 INFO [Listener at localhost/35085 {}] client.AsyncConnectionImpl(233): Connection has been closed by Listener at localhost/35085. 2023-10-15T16:04:32,549 DEBUG [Listener at localhost/35085 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hbase.thirdparty.com.google.common.io.Closeables.close(Closeables.java:79) at org.apache.hadoop.hbase.HBaseTestingUtil.closeConnection(HBaseTestingUtil.java:2631) at org.apache.hadoop.hbase.HBaseTestingUtil.cleanup(HBaseTestingUtil.java:1087) at org.apache.hadoop.hbase.HBaseTestingUtil.shutdownMiniHBaseCluster(HBaseTestingUtil.java:1056) at org.apache.hadoop.hbase.HBaseTestingUtil.shutdownMiniCluster(HBaseTestingUtil.java:1042) at org.apache.hadoop.hbase.replication.TestSyncReplicationStandbyKillRS.testStandbyKillRegionServer(TestSyncReplicationStandbyKillRS.java:67) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:32,549 DEBUG [Listener at localhost/35085 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:32,550 DEBUG [Listener at localhost/35085 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:32,550 INFO [Registry-endpoints-refresh-end-points {}] client.RegistryEndpointsRefresher(78): Registry end points refresher loop exited. 2023-10-15T16:04:32,550 DEBUG [Listener at localhost/35085 {}] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-10-15T16:04:32,551 INFO [Listener at localhost/35085 {}] master.HMaster(3231): ***** STOPPING master '8c5d72df62de,45299,1697385850462' ***** 2023-10-15T16:04:32,551 INFO [Listener at localhost/35085 {}] master.HMaster(3233): STOPPED: Stopped by Listener at localhost/35085 2023-10-15T16:04:32,551 INFO [M:0;8c5d72df62de:45299 {}] hbase.HBaseServerBase(435): Shutdown chores and chore service 2023-10-15T16:04:32,551 INFO [M:0;8c5d72df62de:45299 {}] hbase.ChoreService(370): Chore service for: master/8c5d72df62de:0 had [] on shutdown 2023-10-15T16:04:32,551 INFO [M:0;8c5d72df62de:45299 {}] hbase.HBaseServerBase(445): Shutdown executor service 2023-10-15T16:04:32,551 DEBUG [M:0;8c5d72df62de:45299 {}] master.HMaster(1759): Stopping service threads 2023-10-15T16:04:32,552 INFO [Listener at localhost/35085 {}] util.JVMClusterUtil(250): Stopped backup Master 252360943 is stopped: true 2023-10-15T16:04:32,553 INFO [M:0;8c5d72df62de:45299 {}] ipc.NettyRpcServer(340): Stopping server on /172.17.0.3:45299 2023-10-15T16:04:32,553 DEBUG [Listener at localhost/35085 {}] util.JVMClusterUtil(257): Found active master hash=650219633, stopped=false 2023-10-15T16:04:32,554 INFO [Listener at localhost/35085 {}] master.ServerManager(892): Cluster shutdown requested of master=8c5d72df62de,41825,1697385851376 2023-10-15T16:04:32,556 INFO [M:0;8c5d72df62de:45299 {}] hbase.HBaseServerBase(476): Close zookeeper 2023-10-15T16:04:32,557 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:40585-0x101b6f97f510003, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster1/running 2023-10-15T16:04:32,557 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster1/running 2023-10-15T16:04:32,557 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:40585-0x101b6f97f510003, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1 2023-10-15T16:04:32,557 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1 2023-10-15T16:04:32,557 INFO [Listener at localhost/35085 {}] procedure2.ProcedureExecutor(630): Stopping 2023-10-15T16:04:32,558 INFO [Listener at localhost/35085 {}] client.AsyncConnectionImpl(233): Connection has been closed by Listener at localhost/35085. 2023-10-15T16:04:32,558 DEBUG [Listener at localhost/35085 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.master.HMaster.lambda$shutdown$17(HMaster.java:3216) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.master.HMaster.shutdown(HMaster.java:3187) at org.apache.hadoop.hbase.util.JVMClusterUtil.shutdown(JVMClusterUtil.java:265) at org.apache.hadoop.hbase.LocalHBaseCluster.shutdown(LocalHBaseCluster.java:416) at org.apache.hadoop.hbase.SingleProcessHBaseCluster.shutdown(SingleProcessHBaseCluster.java:676) at org.apache.hadoop.hbase.HBaseTestingUtil.shutdownMiniHBaseCluster(HBaseTestingUtil.java:1058) at org.apache.hadoop.hbase.HBaseTestingUtil.shutdownMiniCluster(HBaseTestingUtil.java:1042) at org.apache.hadoop.hbase.replication.TestSyncReplicationStandbyKillRS.testStandbyKillRegionServer(TestSyncReplicationStandbyKillRS.java:67) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:32,558 DEBUG [Listener at localhost/35085 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:32,558 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:34443-0x101b6f97f510002, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster1/running 2023-10-15T16:04:32,558 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:34443-0x101b6f97f510002, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1 2023-10-15T16:04:32,558 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:41815-0x101b6f97f510004, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster1/running 2023-10-15T16:04:32,558 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:41815-0x101b6f97f510004, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1 2023-10-15T16:04:32,558 INFO [Listener at localhost/35085 {}] regionserver.HRegionServer(2147): ***** STOPPING region server '8c5d72df62de,34443,1697385851561' ***** 2023-10-15T16:04:32,558 INFO [Listener at localhost/35085 {}] regionserver.HRegionServer(2161): STOPPED: Shutdown requested 2023-10-15T16:04:32,558 INFO [Listener at localhost/35085 {}] regionserver.HRegionServer(2147): ***** STOPPING region server '8c5d72df62de,40585,1697385851811' ***** 2023-10-15T16:04:32,558 INFO [Listener at localhost/35085 {}] regionserver.HRegionServer(2161): STOPPED: Shutdown requested 2023-10-15T16:04:32,558 INFO [Listener at localhost/35085 {}] regionserver.HRegionServer(2147): ***** STOPPING region server '8c5d72df62de,41815,1697385851922' ***** 2023-10-15T16:04:32,558 INFO [Listener at localhost/35085 {}] regionserver.HRegionServer(2161): STOPPED: Shutdown requested 2023-10-15T16:04:32,559 INFO [RS:1;8c5d72df62de:40585 {}] regionserver.HeapMemoryManager(220): Stopping 2023-10-15T16:04:32,559 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.HeapMemoryManager(220): Stopping 2023-10-15T16:04:32,559 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.HeapMemoryManager(220): Stopping 2023-10-15T16:04:32,559 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:45299-0x101b6f97f510000, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster1/running 2023-10-15T16:04:32,559 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:45299-0x101b6f97f510000, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1 2023-10-15T16:04:32,559 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:40585-0x101b6f97f510003, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1/backup-masters 2023-10-15T16:04:32,559 INFO [MemStoreFlusher.0 {}] regionserver.MemStoreFlusher$FlushHandler(374): MemStoreFlusher.0 exiting 2023-10-15T16:04:32,559 INFO [MemStoreFlusher.0 {}] regionserver.MemStoreFlusher$FlushHandler(374): MemStoreFlusher.0 exiting 2023-10-15T16:04:32,559 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:41815-0x101b6f97f510004, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1/backup-masters 2023-10-15T16:04:32,560 INFO [MemStoreFlusher.0 {}] regionserver.MemStoreFlusher$FlushHandler(374): MemStoreFlusher.0 exiting 2023-10-15T16:04:32,560 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1/backup-masters 2023-10-15T16:04:32,560 INFO [RS:2;8c5d72df62de:41815 {}] flush.RegionServerFlushTableProcedureManager(119): Stopping region server flush procedure manager gracefully. 2023-10-15T16:04:32,561 INFO [RS:2;8c5d72df62de:41815 {}] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-10-15T16:04:32,561 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.HRegionServer(3035): Received CLOSE for 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:32,560 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKUtil(113): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Set watcher on znode that does not yet exist, /cluster1/running 2023-10-15T16:04:32,561 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.HRegionServer(943): stopping server 8c5d72df62de,41815,1697385851922 2023-10-15T16:04:32,561 INFO [RS:2;8c5d72df62de:41815 {}] hbase.HBaseServerBase(452): Close async cluster connection 2023-10-15T16:04:32,561 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] hbase.ScheduledChore(149): Chore: MemstoreFlusherChore was stopped 2023-10-15T16:04:32,561 INFO [RS:2;8c5d72df62de:41815 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:2;8c5d72df62de:41815. 2023-10-15T16:04:32,561 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] hbase.ScheduledChore(149): Chore: CompactionChecker was stopped 2023-10-15T16:04:32,561 DEBUG [RS:2;8c5d72df62de:41815 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.HBaseServerBase.closeClusterConnection(HBaseServerBase.java:454) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:946) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:32,561 DEBUG [RS:2;8c5d72df62de:41815 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:32,562 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.CompactSplit(469): Waiting for Split Thread to finish... 2023-10-15T16:04:32,562 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.CompactSplit(469): Waiting for Large Compaction Thread to finish... 2023-10-15T16:04:32,562 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.CompactSplit(469): Waiting for Small Compaction Thread to finish... 2023-10-15T16:04:32,562 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.HRegionServer(3035): Received CLOSE for 1588230740 2023-10-15T16:04:32,560 INFO [RS:0;8c5d72df62de:34443 {}] flush.RegionServerFlushTableProcedureManager(119): Stopping region server flush procedure manager gracefully. 2023-10-15T16:04:32,560 INFO [RS:1;8c5d72df62de:40585 {}] flush.RegionServerFlushTableProcedureManager(119): Stopping region server flush procedure manager gracefully. 2023-10-15T16:04:32,562 INFO [RS:1;8c5d72df62de:40585 {}] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-10-15T16:04:32,562 INFO [RS:0;8c5d72df62de:34443 {}] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-10-15T16:04:32,562 INFO [RS:1;8c5d72df62de:40585 {}] regionserver.HRegionServer(943): stopping server 8c5d72df62de,40585,1697385851811 2023-10-15T16:04:32,562 INFO [RS:1;8c5d72df62de:40585 {}] hbase.HBaseServerBase(452): Close async cluster connection 2023-10-15T16:04:32,562 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.HRegionServer(3035): Received CLOSE for b74afe9e0ba97e30b7723e2e921963b9 2023-10-15T16:04:32,562 INFO [RS:1;8c5d72df62de:40585 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:1;8c5d72df62de:40585. 2023-10-15T16:04:32,562 DEBUG [RS:1;8c5d72df62de:40585 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.HBaseServerBase.closeClusterConnection(HBaseServerBase.java:454) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:946) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:32,562 DEBUG [RS:1;8c5d72df62de:40585 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:32,562 INFO [RS:1;8c5d72df62de:40585 {}] regionserver.HRegionServer(960): stopping server 8c5d72df62de,40585,1697385851811; all regions closed. 2023-10-15T16:04:32,564 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.HRegionServer(1296): Waiting on 2 regions to close 2023-10-15T16:04:32,564 DEBUG [RS:2;8c5d72df62de:41815 {}] regionserver.HRegionServer(1300): Online Regions={1588230740=hbase:meta,,1.1588230740, 9352e165439825b87cc19da7b1c32db3=SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3.} 2023-10-15T16:04:32,564 DEBUG [RS:2;8c5d72df62de:41815 {}] regionserver.HRegionServer(1326): Waiting on 1588230740, 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:32,566 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:45299-0x101b6f97f510000, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster1/backup-masters/8c5d72df62de,45299,1697385850462 2023-10-15T16:04:32,566 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:45299-0x101b6f97f510000, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1/backup-masters 2023-10-15T16:04:32,566 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKUtil(113): regionserver:34443-0x101b6f97f510002, quorum=127.0.0.1:52753, baseZNode=/cluster1 Set watcher on znode that does not yet exist, /cluster1/running 2023-10-15T16:04:32,567 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:34443-0x101b6f97f510002, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1/backup-masters 2023-10-15T16:04:32,567 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKUtil(113): regionserver:40585-0x101b6f97f510003, quorum=127.0.0.1:52753, baseZNode=/cluster1 Set watcher on znode that does not yet exist, /cluster1/running 2023-10-15T16:04:32,567 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKUtil(113): regionserver:41815-0x101b6f97f510004, quorum=127.0.0.1:52753, baseZNode=/cluster1 Set watcher on znode that does not yet exist, /cluster1/running 2023-10-15T16:04:32,573 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.HRegionServer(943): stopping server 8c5d72df62de,34443,1697385851561 2023-10-15T16:04:32,573 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1710): Closing 1588230740, disabling compactions & flushes 2023-10-15T16:04:32,573 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1710): Closing 9352e165439825b87cc19da7b1c32db3, disabling compactions & flushes 2023-10-15T16:04:32,573 INFO [RS:0;8c5d72df62de:34443 {}] hbase.HBaseServerBase(452): Close async cluster connection 2023-10-15T16:04:32,573 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1743): Closing region hbase:meta,,1.1588230740 2023-10-15T16:04:32,573 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1743): Closing region SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:32,573 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1764): Time limited wait for close lock on hbase:meta,,1.1588230740 2023-10-15T16:04:32,573 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1764): Time limited wait for close lock on SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:32,573 INFO [RS:0;8c5d72df62de:34443 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:0;8c5d72df62de:34443. 2023-10-15T16:04:32,574 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1831): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-10-15T16:04:32,574 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1831): Acquired close lock on SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. after waiting 0 ms 2023-10-15T16:04:32,574 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1841): Updates disabled for region hbase:meta,,1.1588230740 2023-10-15T16:04:32,574 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1841): Updates disabled for region SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:32,574 DEBUG [RS:0;8c5d72df62de:34443 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.HBaseServerBase.closeClusterConnection(HBaseServerBase.java:454) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:946) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:32,574 DEBUG [RS:0;8c5d72df62de:34443 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:32,574 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.HRegionServer(1296): Waiting on 1 regions to close 2023-10-15T16:04:32,574 DEBUG [RS:0;8c5d72df62de:34443 {}] regionserver.HRegionServer(1300): Online Regions={b74afe9e0ba97e30b7723e2e921963b9=hbase:replication,,1697385863736.b74afe9e0ba97e30b7723e2e921963b9.} 2023-10-15T16:04:32,574 DEBUG [RS:0;8c5d72df62de:34443 {}] regionserver.HRegionServer(1326): Waiting on b74afe9e0ba97e30b7723e2e921963b9 2023-10-15T16:04:32,580 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1710): Closing b74afe9e0ba97e30b7723e2e921963b9, disabling compactions & flushes 2023-10-15T16:04:32,581 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1743): Closing region hbase:replication,,1697385863736.b74afe9e0ba97e30b7723e2e921963b9. 2023-10-15T16:04:32,581 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1764): Time limited wait for close lock on hbase:replication,,1697385863736.b74afe9e0ba97e30b7723e2e921963b9. 2023-10-15T16:04:32,581 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1831): Acquired close lock on hbase:replication,,1697385863736.b74afe9e0ba97e30b7723e2e921963b9. after waiting 0 ms 2023-10-15T16:04:32,581 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1841): Updates disabled for region hbase:replication,,1697385863736.b74afe9e0ba97e30b7723e2e921963b9. 2023-10-15T16:04:32,581 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(2885): Flushing 9352e165439825b87cc19da7b1c32db3 1/1 column families, dataSize=31.25 KB heapSize=101.81 KB 2023-10-15T16:04:32,584 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(2885): Flushing 1588230740 4/4 column families, dataSize=4.09 KB heapSize=8.12 KB 2023-10-15T16:04:32,595 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(2885): Flushing b74afe9e0ba97e30b7723e2e921963b9 3/3 column families, dataSize=928 B heapSize=2.10 KB 2023-10-15T16:04:32,601 DEBUG [RS:1;8c5d72df62de:40585 {}] wal.AbstractFSWAL(1244): Moved 1 WAL file(s) to /user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/oldWALs 2023-10-15T16:04:32,601 INFO [RS:1;8c5d72df62de:40585 {}] wal.AbstractFSWAL(1247): Closed WAL: AsyncFSWAL 8c5d72df62de%2C40585%2C1697385851811:(num 1697385854701) 2023-10-15T16:04:32,601 DEBUG [RS:1;8c5d72df62de:40585 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:32,601 INFO [RS:1;8c5d72df62de:40585 {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:32,601 INFO [RS:1;8c5d72df62de:40585 {}] hbase.HBaseServerBase(435): Shutdown chores and chore service 2023-10-15T16:04:32,604 INFO [RS:1;8c5d72df62de:40585 {}] hbase.ChoreService(370): Chore service for: regionserver/8c5d72df62de:0 had [ScheduledChore name=ReplicationSourceStatistics, period=5000, unit=MILLISECONDS, ScheduledChore name=ReplicationSinkStatistics, period=5000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-10-15T16:04:32,604 INFO [RS:1;8c5d72df62de:40585 {}] regionserver.CompactSplit(469): Waiting for Split Thread to finish... 2023-10-15T16:04:32,604 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(249): LogRoller exiting. 2023-10-15T16:04:32,604 INFO [RS:1;8c5d72df62de:40585 {}] regionserver.CompactSplit(469): Waiting for Large Compaction Thread to finish... 2023-10-15T16:04:32,604 INFO [RS:1;8c5d72df62de:40585 {}] regionserver.CompactSplit(469): Waiting for Small Compaction Thread to finish... 2023-10-15T16:04:32,604 INFO [RS:1;8c5d72df62de:40585 {}] hbase.HBaseServerBase(445): Shutdown executor service 2023-10-15T16:04:32,604 INFO [RS:1;8c5d72df62de:40585 {}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,40585,1697385851811 because: Region server is closing 2023-10-15T16:04:32,621 INFO [regionserver/8c5d72df62de:0.leaseChecker {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:32,621 INFO [regionserver/8c5d72df62de:0.leaseChecker {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:32,651 INFO [regionserver/8c5d72df62de:0.leaseChecker {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:32,666 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:45299-0x101b6f97f510000, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:32,666 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:45299-0x101b6f97f510000, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:32,666 INFO [M:0;8c5d72df62de:45299 {}] hbase.HBaseServerBase(483): Close table descriptors 2023-10-15T16:04:32,694 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/replication/b74afe9e0ba97e30b7723e2e921963b9/.tmp/queue/9e8651ee1dfb43779eb5853b0f10d3c0 is 193, key is 1-8c5d72df62de,41815,1697385851922/queue:8c5d72df62de%2C41815%2C1697385851922-1697385868530-1/1697385869132/Put/seqid=0 2023-10-15T16:04:32,694 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/info/95dc8fb6f82345f7931dd2606f6ef14f is 147, key is hbase:replication,,1697385863736.b74afe9e0ba97e30b7723e2e921963b9./info:regioninfo/1697385864205/Put/seqid=0 2023-10-15T16:04:32,704 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/default/SyncRep/9352e165439825b87cc19da7b1c32db3/.tmp/cf/2c93509345e8464998fa0e9fc2c3ed02 is 36, key is \x00\x00\x00\x00/cf:cq/1697385870586/Put/seqid=0 2023-10-15T16:04:32,726 WARN [HBase-Metrics2-1 {}] impl.MetricsConfig(136): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-10-15T16:04:32,733 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=928 B at sequenceid=10 (bloomFilter=true), to=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/replication/b74afe9e0ba97e30b7723e2e921963b9/.tmp/queue/9e8651ee1dfb43779eb5853b0f10d3c0 2023-10-15T16:04:32,737 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=31.25 KB at sequenceid=1007 (bloomFilter=true), to=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/default/SyncRep/9352e165439825b87cc19da7b1c32db3/.tmp/cf/2c93509345e8464998fa0e9fc2c3ed02 2023-10-15T16:04:32,745 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=3.63 KB at sequenceid=20 (bloomFilter=true), to=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/info/95dc8fb6f82345f7931dd2606f6ef14f 2023-10-15T16:04:32,764 DEBUG [RS:2;8c5d72df62de:41815 {}] regionserver.HRegionServer(1326): Waiting on 1588230740, 9352e165439825b87cc19da7b1c32db3 2023-10-15T16:04:32,768 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f510026, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:32,768 INFO [RS:1;8c5d72df62de:40585 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:1;8c5d72df62de:40585. 2023-10-15T16:04:32,768 DEBUG [RS:1;8c5d72df62de:40585 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:664) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:659) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:987) at org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:160) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2481) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:995) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:32,768 DEBUG [RS:1;8c5d72df62de:40585 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:32,768 DEBUG [RS:1;8c5d72df62de:40585 {}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x1fc01338 to 127.0.0.1:52753 2023-10-15T16:04:32,768 INFO [RS:1;8c5d72df62de:40585 {}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,40585,1697385851811.replicationSource.shipper8c5d72df62de%2C40585%2C1697385851811,1-8c5d72df62de,40585,1697385851811 terminated 2023-10-15T16:04:32,776 DEBUG [RS:0;8c5d72df62de:34443 {}] regionserver.HRegionServer(1326): Waiting on b74afe9e0ba97e30b7723e2e921963b9 2023-10-15T16:04:32,777 INFO [RS:1;8c5d72df62de:40585 {}] ipc.NettyRpcServer(340): Stopping server on /172.17.0.3:40585 2023-10-15T16:04:32,788 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] util.BlockIOUtils(63): Unable to find positioned bytebuffer read API of FSDataInputStream. preadWithExtra() will use a temporary on-heap byte array. 2023-10-15T16:04:32,828 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/replication/b74afe9e0ba97e30b7723e2e921963b9/.tmp/queue/9e8651ee1dfb43779eb5853b0f10d3c0 as hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/replication/b74afe9e0ba97e30b7723e2e921963b9/queue/9e8651ee1dfb43779eb5853b0f10d3c0 2023-10-15T16:04:32,829 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/default/SyncRep/9352e165439825b87cc19da7b1c32db3/.tmp/cf/2c93509345e8464998fa0e9fc2c3ed02 as hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/default/SyncRep/9352e165439825b87cc19da7b1c32db3/cf/2c93509345e8464998fa0e9fc2c3ed02 2023-10-15T16:04:32,840 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/default/SyncRep/9352e165439825b87cc19da7b1c32db3/cf/2c93509345e8464998fa0e9fc2c3ed02, entries=1000, sequenceid=1007, filesize=40.6 K 2023-10-15T16:04:32,841 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/replication/b74afe9e0ba97e30b7723e2e921963b9/queue/9e8651ee1dfb43779eb5853b0f10d3c0, entries=4, sequenceid=10, filesize=5.7 K 2023-10-15T16:04:32,846 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(3123): Finished flush of dataSize ~31.25 KB/32000, heapSize ~101.80 KB/104240, currentSize=0 B/0 for 9352e165439825b87cc19da7b1c32db3 in 264ms, sequenceid=1007, compaction requested=false 2023-10-15T16:04:32,846 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(3123): Finished flush of dataSize ~928 B/928, heapSize ~1.59 KB/1624, currentSize=0 B/0 for b74afe9e0ba97e30b7723e2e921963b9 in 256ms, sequenceid=10, compaction requested=false 2023-10-15T16:04:32,851 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/ns/da453a7c3a8e46b1aac2323f9aaa55cf is 43, key is default/ns:d/1697385855705/Put/seqid=0 2023-10-15T16:04:32,861 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] wal.WALSplitUtil(410): Wrote file=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/replication/b74afe9e0ba97e30b7723e2e921963b9/recovered.edits/13.seqid, newMaxSeqId=13, maxSeqId=1 2023-10-15T16:04:32,861 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] wal.WALSplitUtil(410): Wrote file=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/default/SyncRep/9352e165439825b87cc19da7b1c32db3/recovered.edits/1010.seqid, newMaxSeqId=1010, maxSeqId=4 2023-10-15T16:04:32,863 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1961): Closed SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:32,863 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1664): Region close journal for 9352e165439825b87cc19da7b1c32db3: Waiting for close lock at 1697385872561Running coprocessor pre-close hooks at 1697385872562 (+1 ms)Disabling compacts and flushes for region at 1697385872562Disabling writes for close at 1697385872574 (+12 ms)Obtaining lock to block concurrent updates at 1697385872581 (+7 ms)Preparing flush snapshotting stores in 9352e165439825b87cc19da7b1c32db3 at 1697385872581Finished memstore snapshotting SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3., syncing WAL and waiting on mvcc, flushsize=dataSize=32000, getHeapSize=104240, getOffHeapSize=0, getCellsCount=1000 at 1697385872606 (+25 ms)Flushing stores of SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. at 1697385872609 (+3 ms)Flushing 9352e165439825b87cc19da7b1c32db3/cf: creating writer at 1697385872610 (+1 ms)Flushing 9352e165439825b87cc19da7b1c32db3/cf: appending metadata at 1697385872701 (+91 ms)Flushing 9352e165439825b87cc19da7b1c32db3/cf: closing flushed file at 1697385872701Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@5053751a: reopening flushed file at 1697385872827 (+126 ms)Finished flush of dataSize ~31.25 KB/32000, heapSize ~101.80 KB/104240, currentSize=0 B/0 for 9352e165439825b87cc19da7b1c32db3 in 264ms, sequenceid=1007, compaction requested=false at 1697385872846 (+19 ms)Writing region close event to WAL at 1697385872855 (+9 ms)Running coprocessor post-close hooks at 1697385872862 (+7 ms)Closed at 1697385872863 (+1 ms) 2023-10-15T16:04:32,863 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-15T16:04:32,863 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] handler.CloseRegionHandler(117): Closed SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3. 2023-10-15T16:04:32,864 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1961): Closed hbase:replication,,1697385863736.b74afe9e0ba97e30b7723e2e921963b9. 2023-10-15T16:04:32,864 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1664): Region close journal for b74afe9e0ba97e30b7723e2e921963b9: Waiting for close lock at 1697385872574Running coprocessor pre-close hooks at 1697385872574Disabling compacts and flushes for region at 1697385872574Disabling writes for close at 1697385872581 (+7 ms)Obtaining lock to block concurrent updates at 1697385872595 (+14 ms)Preparing flush snapshotting stores in b74afe9e0ba97e30b7723e2e921963b9 at 1697385872595Finished memstore snapshotting hbase:replication,,1697385863736.b74afe9e0ba97e30b7723e2e921963b9., syncing WAL and waiting on mvcc, flushsize=dataSize=928, getHeapSize=2104, getOffHeapSize=0, getCellsCount=6 at 1697385872606 (+11 ms)Flushing stores of hbase:replication,,1697385863736.b74afe9e0ba97e30b7723e2e921963b9. at 1697385872609 (+3 ms)Flushing b74afe9e0ba97e30b7723e2e921963b9/queue: creating writer at 1697385872610 (+1 ms)Flushing b74afe9e0ba97e30b7723e2e921963b9/queue: appending metadata at 1697385872674 (+64 ms)Flushing b74afe9e0ba97e30b7723e2e921963b9/queue: closing flushed file at 1697385872678 (+4 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@e17cb23: reopening flushed file at 1697385872827 (+149 ms)Finished flush of dataSize ~928 B/928, heapSize ~1.59 KB/1624, currentSize=0 B/0 for b74afe9e0ba97e30b7723e2e921963b9 in 256ms, sequenceid=10, compaction requested=false at 1697385872846 (+19 ms)Writing region close event to WAL at 1697385872855 (+9 ms)Running coprocessor post-close hooks at 1697385872863 (+8 ms)Closed at 1697385872864 (+1 ms) 2023-10-15T16:04:32,864 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] handler.CloseRegionHandler(117): Closed hbase:replication,,1697385863736.b74afe9e0ba97e30b7723e2e921963b9. 2023-10-15T16:04:32,873 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1/rs 2023-10-15T16:04:32,873 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:40585-0x101b6f97f510003, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster1/rs/8c5d72df62de,40585,1697385851811 2023-10-15T16:04:32,873 INFO [RS:1;8c5d72df62de:40585 {}] hbase.HBaseServerBase(476): Close zookeeper 2023-10-15T16:04:32,877 INFO [RegionServerTracker-0 {}] master.RegionServerTracker(171): RegionServer ephemeral node deleted, processing expiration [8c5d72df62de,40585,1697385851811] 2023-10-15T16:04:32,878 DEBUG [RegionServerTracker-0 {}] zookeeper.RecoverableZooKeeper(221): Node /cluster1/draining/8c5d72df62de,40585,1697385851811 already deleted, retry=false 2023-10-15T16:04:32,878 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=74 B at sequenceid=20 (bloomFilter=true), to=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/ns/da453a7c3a8e46b1aac2323f9aaa55cf 2023-10-15T16:04:32,878 INFO [RegionServerTracker-0 {}] master.ServerManager(597): Cluster shutdown set; 8c5d72df62de,40585,1697385851811 expired; onlineServers=2 2023-10-15T16:04:32,899 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/rep_barrier/693fd8d5a4df48269ec14573b45f110b is 115, key is SyncRep,,1697385861726.9352e165439825b87cc19da7b1c32db3./rep_barrier:seqnumDuringOpen/1697385869004/Put/seqid=0 2023-10-15T16:04:32,909 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=222 B at sequenceid=20 (bloomFilter=true), to=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/rep_barrier/693fd8d5a4df48269ec14573b45f110b 2023-10-15T16:04:32,973 DEBUG [RS:2;8c5d72df62de:41815 {}] regionserver.HRegionServer(1326): Waiting on 1588230740 2023-10-15T16:04:32,976 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:40585-0x101b6f97f510003, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:32,976 INFO [RS:1;8c5d72df62de:40585 {}] hbase.HBaseServerBase(483): Close table descriptors 2023-10-15T16:04:32,976 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:40585-0x101b6f97f510003, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:32,976 INFO [RS:1;8c5d72df62de:40585 {}] regionserver.HRegionServer(1015): Exiting; stopping=8c5d72df62de,40585,1697385851811; zookeeper connection closed. 2023-10-15T16:04:32,976 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3e8016f9 {}] hbase.SingleProcessHBaseCluster$SingleFileSystemShutdownThread(211): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3e8016f9 2023-10-15T16:04:32,976 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.HRegionServer(960): stopping server 8c5d72df62de,34443,1697385851561; all regions closed. 2023-10-15T16:04:32,986 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/table/9eadc02ba86c448ea31d783a4327763c is 53, key is hbase:replication/table:state/1697385864218/Put/seqid=0 2023-10-15T16:04:32,993 WARN [Close-WAL-Writer-0 {}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(596): complete file /user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/WALs/8c5d72df62de,34443,1697385851561/8c5d72df62de%2C34443%2C1697385851561.rep.1697385864145 not finished, retry = 0 2023-10-15T16:04:33,007 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=176 B at sequenceid=20 (bloomFilter=true), to=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/table/9eadc02ba86c448ea31d783a4327763c 2023-10-15T16:04:33,015 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/info/95dc8fb6f82345f7931dd2606f6ef14f as hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/info/95dc8fb6f82345f7931dd2606f6ef14f 2023-10-15T16:04:33,022 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/info/95dc8fb6f82345f7931dd2606f6ef14f, entries=25, sequenceid=20, filesize=8.1 K 2023-10-15T16:04:33,024 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/ns/da453a7c3a8e46b1aac2323f9aaa55cf as hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/ns/da453a7c3a8e46b1aac2323f9aaa55cf 2023-10-15T16:04:33,031 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/ns/da453a7c3a8e46b1aac2323f9aaa55cf, entries=2, sequenceid=20, filesize=5.0 K 2023-10-15T16:04:33,033 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/rep_barrier/693fd8d5a4df48269ec14573b45f110b as hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/rep_barrier/693fd8d5a4df48269ec14573b45f110b 2023-10-15T16:04:33,040 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/rep_barrier/693fd8d5a4df48269ec14573b45f110b, entries=2, sequenceid=20, filesize=5.5 K 2023-10-15T16:04:33,041 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/.tmp/table/9eadc02ba86c448ea31d783a4327763c as hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/table/9eadc02ba86c448ea31d783a4327763c 2023-10-15T16:04:33,048 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/table/9eadc02ba86c448ea31d783a4327763c, entries=4, sequenceid=20, filesize=5.2 K 2023-10-15T16:04:33,050 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(3123): Finished flush of dataSize ~4.09 KB/4185, heapSize ~8.05 KB/8248, currentSize=0 B/0 for 1588230740 in 468ms, sequenceid=20, compaction requested=false 2023-10-15T16:04:33,063 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] wal.WALSplitUtil(410): Wrote file=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/data/hbase/meta/1588230740/recovered.edits/23.seqid, newMaxSeqId=23, maxSeqId=1 2023-10-15T16:04:33,063 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-15T16:04:33,064 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1961): Closed hbase:meta,,1.1588230740 2023-10-15T16:04:33,064 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1664): Region close journal for 1588230740: Waiting for close lock at 1697385872564Running coprocessor pre-close hooks at 1697385872564Disabling compacts and flushes for region at 1697385872564Disabling writes for close at 1697385872574 (+10 ms)Obtaining lock to block concurrent updates at 1697385872584 (+10 ms)Preparing flush snapshotting stores in 1588230740 at 1697385872584Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=4185, getHeapSize=8248, getOffHeapSize=0, getCellsCount=41 at 1697385872606 (+22 ms)Flushing stores of hbase:meta,,1.1588230740 at 1697385872609 (+3 ms)Flushing 1588230740/info: creating writer at 1697385872610 (+1 ms)Flushing 1588230740/info: appending metadata at 1697385872675 (+65 ms)Flushing 1588230740/info: closing flushed file at 1697385872678 (+3 ms)Flushing 1588230740/ns: creating writer at 1697385872827 (+149 ms)Flushing 1588230740/ns: appending metadata at 1697385872850 (+23 ms)Flushing 1588230740/ns: closing flushed file at 1697385872850Flushing 1588230740/rep_barrier: creating writer at 1697385872884 (+34 ms)Flushing 1588230740/rep_barrier: appending metadata at 1697385872898 (+14 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1697385872899 (+1 ms)Flushing 1588230740/table: creating writer at 1697385872915 (+16 ms)Flushing 1588230740/table: appending metadata at 1697385872985 (+70 ms)Flushing 1588230740/table: closing flushed file at 1697385872985Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@4088030c: reopening flushed file at 1697385873014 (+29 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@33f0f0e: reopening flushed file at 1697385873023 (+9 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@2d67cbf5: reopening flushed file at 1697385873031 (+8 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@4983abbe: reopening flushed file at 1697385873040 (+9 ms)Finished flush of dataSize ~4.09 KB/4185, heapSize ~8.05 KB/8248, currentSize=0 B/0 for 1588230740 in 468ms, sequenceid=20, compaction requested=false at 1697385873050 (+10 ms)Writing region close event to WAL at 1697385873059 (+9 ms)Running coprocessor post-close hooks at 1697385873063 (+4 ms)Closed at 1697385873064 (+1 ms) 2023-10-15T16:04:33,064 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-10-15T16:04:33,097 DEBUG [RS:0;8c5d72df62de:34443 {}] wal.AbstractFSWAL(1244): Moved 1 WAL file(s) to /user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/oldWALs 2023-10-15T16:04:33,097 INFO [RS:0;8c5d72df62de:34443 {}] wal.AbstractFSWAL(1247): Closed WAL: AsyncFSWAL 8c5d72df62de%2C34443%2C1697385851561.rep:(num 1697385864145) 2023-10-15T16:04:33,105 DEBUG [RS:0;8c5d72df62de:34443 {}] wal.AbstractFSWAL(1244): Moved 1 WAL file(s) to /user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/oldWALs 2023-10-15T16:04:33,105 INFO [RS:0;8c5d72df62de:34443 {}] wal.AbstractFSWAL(1247): Closed WAL: AsyncFSWAL 8c5d72df62de%2C34443%2C1697385851561:(num 1697385854712) 2023-10-15T16:04:33,105 DEBUG [RS:0;8c5d72df62de:34443 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:33,105 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:33,105 INFO [RS:0;8c5d72df62de:34443 {}] hbase.HBaseServerBase(435): Shutdown chores and chore service 2023-10-15T16:04:33,106 INFO [RS:0;8c5d72df62de:34443 {}] hbase.ChoreService(370): Chore service for: regionserver/8c5d72df62de:0 had [ScheduledChore name=ReplicationSourceStatistics, period=5000, unit=MILLISECONDS, ScheduledChore name=ReplicationSinkStatistics, period=5000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-10-15T16:04:33,106 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.CompactSplit(469): Waiting for Split Thread to finish... 2023-10-15T16:04:33,106 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.CompactSplit(469): Waiting for Large Compaction Thread to finish... 2023-10-15T16:04:33,106 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(249): LogRoller exiting. 2023-10-15T16:04:33,106 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.CompactSplit(469): Waiting for Small Compaction Thread to finish... 2023-10-15T16:04:33,106 INFO [RS:0;8c5d72df62de:34443 {}] hbase.HBaseServerBase(445): Shutdown executor service 2023-10-15T16:04:33,106 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,34443,1697385851561 because: Region server is closing 2023-10-15T16:04:33,173 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.HRegionServer(960): stopping server 8c5d72df62de,41815,1697385851922; all regions closed. 2023-10-15T16:04:33,181 DEBUG [RS:2;8c5d72df62de:41815 {}] wal.AbstractFSWAL(1244): Moved 1 WAL file(s) to /user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/oldWALs 2023-10-15T16:04:33,181 INFO [RS:2;8c5d72df62de:41815 {}] wal.AbstractFSWAL(1247): Closed WAL: AsyncFSWAL 8c5d72df62de%2C41815%2C1697385851922.meta:.meta(num 1697385855319) 2023-10-15T16:04:33,191 DEBUG [RS:2;8c5d72df62de:41815 {}] wal.AbstractFSWAL(1244): Moved 1 WAL file(s) to /user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/oldWALs 2023-10-15T16:04:33,191 INFO [RS:2;8c5d72df62de:41815 {}] wal.AbstractFSWAL(1247): Closed WAL: AsyncFSWAL 8c5d72df62de%2C41815%2C1697385851922-1697385868530-1:.syncrep(num 1697385868532) 2023-10-15T16:04:33,201 DEBUG [RS:2;8c5d72df62de:41815 {}] wal.AbstractFSWAL(1244): Moved 1 WAL file(s) to /user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/oldWALs 2023-10-15T16:04:33,201 INFO [RS:2;8c5d72df62de:41815 {}] wal.AbstractFSWAL(1247): Closed WAL: AsyncFSWAL 8c5d72df62de%2C41815%2C1697385851922:(num 1697385854699) 2023-10-15T16:04:33,201 DEBUG [RS:2;8c5d72df62de:41815 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:33,201 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:33,201 INFO [RS:2;8c5d72df62de:41815 {}] hbase.HBaseServerBase(435): Shutdown chores and chore service 2023-10-15T16:04:33,201 INFO [RS:2;8c5d72df62de:41815 {}] hbase.ChoreService(370): Chore service for: regionserver/8c5d72df62de:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=ReplicationSourceStatistics, period=5000, unit=MILLISECONDS, ScheduledChore name=ReplicationSinkStatistics, period=5000, unit=MILLISECONDS] on shutdown 2023-10-15T16:04:33,202 INFO [RS:2;8c5d72df62de:41815 {}] hbase.HBaseServerBase(445): Shutdown executor service 2023-10-15T16:04:33,202 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(249): LogRoller exiting. 2023-10-15T16:04:33,202 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,41815,1697385851922 because: Region server is closing 2023-10-15T16:04:33,206 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 0, reset compression=false 2023-10-15T16:04:33,208 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f510024, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:33,209 INFO [RS:0;8c5d72df62de:34443 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:0;8c5d72df62de:34443. 2023-10-15T16:04:33,209 DEBUG [RS:0;8c5d72df62de:34443 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:664) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:659) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:987) at org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:160) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2481) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:995) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:33,209 DEBUG [RS:0;8c5d72df62de:34443 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:33,209 DEBUG [RS:0;8c5d72df62de:34443 {}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x24556a9b to 127.0.0.1:52753 2023-10-15T16:04:33,209 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,34443,1697385851561.replicationSource.shipper8c5d72df62de%2C34443%2C1697385851561,1-8c5d72df62de,34443,1697385851561 terminated 2023-10-15T16:04:33,210 INFO [RS:0;8c5d72df62de:34443 {}] ipc.NettyRpcServer(340): Stopping server on /172.17.0.3:34443 2023-10-15T16:04:33,213 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:33,214 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1/rs 2023-10-15T16:04:33,214 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:34443-0x101b6f97f510002, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster1/rs/8c5d72df62de,34443,1697385851561 2023-10-15T16:04:33,214 INFO [RS:0;8c5d72df62de:34443 {}] hbase.HBaseServerBase(476): Close zookeeper 2023-10-15T16:04:33,216 INFO [RegionServerTracker-0 {}] master.RegionServerTracker(171): RegionServer ephemeral node deleted, processing expiration [8c5d72df62de,34443,1697385851561] 2023-10-15T16:04:33,217 DEBUG [RegionServerTracker-0 {}] zookeeper.RecoverableZooKeeper(221): Node /cluster1/draining/8c5d72df62de,34443,1697385851561 already deleted, retry=false 2023-10-15T16:04:33,217 INFO [RegionServerTracker-0 {}] master.ServerManager(597): Cluster shutdown set; 8c5d72df62de,34443,1697385851561 expired; onlineServers=1 2023-10-15T16:04:33,289 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:33,293 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465 to pos 0, reset compression=false 2023-10-15T16:04:33,294 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:33,297 DEBUG [regionserver/8c5d72df62de:0.logRoller.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=true 2023-10-15T16:04:33,309 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f510028, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:33,309 INFO [RS:2;8c5d72df62de:41815 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:2;8c5d72df62de:41815. 2023-10-15T16:04:33,310 DEBUG [RS:2;8c5d72df62de:41815 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:664) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:659) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:987) at org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:160) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2481) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:995) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:33,310 DEBUG [RS:2;8c5d72df62de:41815 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:33,310 DEBUG [RS:2;8c5d72df62de:41815 {}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x4910ae8c to 127.0.0.1:52753 2023-10-15T16:04:33,310 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.shipper8c5d72df62de%2C41815%2C1697385851922,1-8c5d72df62de,41815,1697385851922 terminated 2023-10-15T16:04:33,310 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,41815,1697385851922.replicationSource.shipper8c5d72df62de%2C41815%2C1697385851922-1697385868530-1,1-8c5d72df62de,41815,1697385851922 terminated 2023-10-15T16:04:33,311 INFO [RS:2;8c5d72df62de:41815 {}] ipc.NettyRpcServer(340): Stopping server on /172.17.0.3:41815 2023-10-15T16:04:33,314 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1/rs 2023-10-15T16:04:33,314 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:41815-0x101b6f97f510004, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster1/rs/8c5d72df62de,41815,1697385851922 2023-10-15T16:04:33,315 INFO [RS:2;8c5d72df62de:41815 {}] hbase.HBaseServerBase(476): Close zookeeper 2023-10-15T16:04:33,315 INFO [RegionServerTracker-0 {}] master.RegionServerTracker(171): RegionServer ephemeral node deleted, processing expiration [8c5d72df62de,41815,1697385851922] 2023-10-15T16:04:33,317 DEBUG [RegionServerTracker-0 {}] zookeeper.RecoverableZooKeeper(221): Node /cluster1/draining/8c5d72df62de,41815,1697385851922 already deleted, retry=false 2023-10-15T16:04:33,317 INFO [RegionServerTracker-0 {}] master.ServerManager(597): Cluster shutdown set; 8c5d72df62de,41815,1697385851922 expired; onlineServers=0 2023-10-15T16:04:33,317 INFO [RegionServerTracker-0 {}] master.HMaster(3231): ***** STOPPING master '8c5d72df62de,41825,1697385851376' ***** 2023-10-15T16:04:33,317 INFO [RegionServerTracker-0 {}] master.HMaster(3233): STOPPED: Cluster shutdown set; onlineServer=0 2023-10-15T16:04:33,317 INFO [M:1;8c5d72df62de:41825 {}] hbase.HBaseServerBase(452): Close async cluster connection 2023-10-15T16:04:33,318 INFO [M:1;8c5d72df62de:41825 {}] hbase.HBaseServerBase(435): Shutdown chores and chore service 2023-10-15T16:04:33,318 DEBUG [M:1;8c5d72df62de:41825 {}] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-10-15T16:04:33,318 WARN [OldWALsCleaner-0 {}] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-10-15T16:04:33,318 DEBUG [M:1;8c5d72df62de:41825 {}] cleaner.HFileCleaner(335): Stopping file delete threads 2023-10-15T16:04:33,318 DEBUG [master/8c5d72df62de:0:becomeActiveMaster-HFileCleaner.large.0-1697385854157 {}] cleaner.HFileCleaner(306): Exit Thread[master/8c5d72df62de:0:becomeActiveMaster-HFileCleaner.large.0-1697385854157,5,FailOnTimeoutGroup] 2023-10-15T16:04:33,318 DEBUG [master/8c5d72df62de:0:becomeActiveMaster-HFileCleaner.small.0-1697385854164 {}] cleaner.HFileCleaner(306): Exit Thread[master/8c5d72df62de:0:becomeActiveMaster-HFileCleaner.small.0-1697385854164,5,FailOnTimeoutGroup] 2023-10-15T16:04:33,318 INFO [M:1;8c5d72df62de:41825 {}] hbase.ChoreService(370): Chore service for: master/8c5d72df62de:0 had [ScheduledChore name=FlushedSequenceIdFlusher, period=10800000, unit=MILLISECONDS] on shutdown 2023-10-15T16:04:33,318 INFO [M:1;8c5d72df62de:41825 {}] hbase.HBaseServerBase(445): Shutdown executor service 2023-10-15T16:04:33,318 DEBUG [M:1;8c5d72df62de:41825 {}] master.HMaster(1759): Stopping service threads 2023-10-15T16:04:33,318 INFO [M:1;8c5d72df62de:41825 {}] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-10-15T16:04:33,319 ERROR [M:1;8c5d72df62de:41825 {}] procedure2.ProcedureExecutor(654): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-10-15T16:04:33,320 INFO [M:1;8c5d72df62de:41825 {}] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-10-15T16:04:33,320 DEBUG [normalizer-worker-0 {}] normalizer.RegionNormalizerWorker(193): interrupt detected. terminating. 2023-10-15T16:04:33,321 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster1/master 2023-10-15T16:04:33,321 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster1 2023-10-15T16:04:33,321 DEBUG [M:1;8c5d72df62de:41825 {}] zookeeper.ZKUtil(347): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Unable to get data of znode /cluster1/master because node does not exist (not an error) 2023-10-15T16:04:33,321 WARN [M:1;8c5d72df62de:41825 {}] master.ActiveMasterManager(344): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-10-15T16:04:33,322 INFO [M:1;8c5d72df62de:41825 {}] master.ServerManager(1048): Writing .lastflushedseqids file at: hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/.lastflushedseqids 2023-10-15T16:04:33,342 INFO [M:1;8c5d72df62de:41825 {}] assignment.AssignmentManager(384): Stopping assignment manager 2023-10-15T16:04:33,342 INFO [M:1;8c5d72df62de:41825 {}] region.MasterRegion(178): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-10-15T16:04:33,343 DEBUG [M:1;8c5d72df62de:41825 {}] regionserver.HRegion(1710): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-10-15T16:04:33,343 INFO [M:1;8c5d72df62de:41825 {}] regionserver.HRegion(1743): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-15T16:04:33,343 DEBUG [M:1;8c5d72df62de:41825 {}] regionserver.HRegion(1764): Time limited wait for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-15T16:04:33,343 DEBUG [M:1;8c5d72df62de:41825 {}] regionserver.HRegion(1831): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-10-15T16:04:33,343 DEBUG [M:1;8c5d72df62de:41825 {}] regionserver.HRegion(1841): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-15T16:04:33,344 INFO [M:1;8c5d72df62de:41825 {}] regionserver.HRegion(2885): Flushing 1595e783b53d99cd5eef43b6debb2682 4/4 column families, dataSize=83.83 KB heapSize=104.07 KB 2023-10-15T16:04:33,363 DEBUG [M:1;8c5d72df62de:41825 {}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/info/a746f61fe92a4053af1c25b81bd8e364 is 82, key is hbase:meta,,1/info:regioninfo/1697385855582/Put/seqid=0 2023-10-15T16:04:33,375 INFO [M:1;8c5d72df62de:41825 {}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=504 B at sequenceid=212 (bloomFilter=true), to=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/info/a746f61fe92a4053af1c25b81bd8e364 2023-10-15T16:04:33,403 DEBUG [M:1;8c5d72df62de:41825 {}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5050a3e75de143dfa550e0f9d7c4db22 is 1475, key is \x00\x00\x00\x00\x00\x00\x00\x08/proc:d/1697385864225/Put/seqid=0 2023-10-15T16:04:33,409 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:34443-0x101b6f97f510002, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:33,409 INFO [RS:0;8c5d72df62de:34443 {}] hbase.HBaseServerBase(483): Close table descriptors 2023-10-15T16:04:33,409 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:34443-0x101b6f97f510002, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:33,409 INFO [RS:0;8c5d72df62de:34443 {}] regionserver.HRegionServer(1015): Exiting; stopping=8c5d72df62de,34443,1697385851561; zookeeper connection closed. 2023-10-15T16:04:33,417 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@17ce8a52 {}] hbase.SingleProcessHBaseCluster$SingleFileSystemShutdownThread(211): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@17ce8a52 2023-10-15T16:04:33,427 INFO [M:1;8c5d72df62de:41825 {}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=83.15 KB at sequenceid=212 (bloomFilter=true), to=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5050a3e75de143dfa550e0f9d7c4db22 2023-10-15T16:04:33,447 DEBUG [M:1;8c5d72df62de:41825 {}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/rs/eab450a976c24995980c1fc2dbcef295 is 69, key is 8c5d72df62de,34443,1697385851561/rs:state/1697385854333/Put/seqid=0 2023-10-15T16:04:33,462 INFO [M:1;8c5d72df62de:41825 {}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=195 B at sequenceid=212 (bloomFilter=true), to=hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/rs/eab450a976c24995980c1fc2dbcef295 2023-10-15T16:04:33,470 DEBUG [M:1;8c5d72df62de:41825 {}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/info/a746f61fe92a4053af1c25b81bd8e364 as hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/info/a746f61fe92a4053af1c25b81bd8e364 2023-10-15T16:04:33,475 INFO [M:1;8c5d72df62de:41825 {}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/info/a746f61fe92a4053af1c25b81bd8e364, entries=8, sequenceid=212, filesize=5.5 K 2023-10-15T16:04:33,476 DEBUG [M:1;8c5d72df62de:41825 {}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5050a3e75de143dfa550e0f9d7c4db22 as hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5050a3e75de143dfa550e0f9d7c4db22 2023-10-15T16:04:33,488 INFO [M:1;8c5d72df62de:41825 {}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5050a3e75de143dfa550e0f9d7c4db22, entries=28, sequenceid=212, filesize=9.1 K 2023-10-15T16:04:33,489 DEBUG [M:1;8c5d72df62de:41825 {}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/rs/eab450a976c24995980c1fc2dbcef295 as hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/rs/eab450a976c24995980c1fc2dbcef295 2023-10-15T16:04:33,494 INFO [M:1;8c5d72df62de:41825 {}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/rs/eab450a976c24995980c1fc2dbcef295, entries=3, sequenceid=212, filesize=5.2 K 2023-10-15T16:04:33,495 INFO [M:1;8c5d72df62de:41825 {}] regionserver.HRegion(3123): Finished flush of dataSize ~83.83 KB/85843, heapSize ~103.77 KB/106264, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 152ms, sequenceid=212, compaction requested=false 2023-10-15T16:04:33,500 INFO [M:1;8c5d72df62de:41825 {}] regionserver.HRegion(1961): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-10-15T16:04:33,500 DEBUG [M:1;8c5d72df62de:41825 {}] regionserver.HRegion(1664): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1697385873343Disabling compacts and flushes for region at 1697385873343Disabling writes for close at 1697385873343Obtaining lock to block concurrent updates at 1697385873344 (+1 ms)Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1697385873344Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=85843, getHeapSize=106504, getOffHeapSize=0, getCellsCount=259 at 1697385873344Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1697385873345 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/info: creating writer at 1697385873345Flushing 1595e783b53d99cd5eef43b6debb2682/info: appending metadata at 1697385873363 (+18 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/info: closing flushed file at 1697385873363Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1697385873381 (+18 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1697385873402 (+21 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1697385873402Flushing 1595e783b53d99cd5eef43b6debb2682/rs: creating writer at 1697385873432 (+30 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/rs: appending metadata at 1697385873447 (+15 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/rs: closing flushed file at 1697385873447Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@3eacc04b: reopening flushed file at 1697385873469 (+22 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@42fe9723: reopening flushed file at 1697385873476 (+7 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@47e2239f: reopening flushed file at 1697385873488 (+12 ms)Finished flush of dataSize ~83.83 KB/85843, heapSize ~103.77 KB/106264, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 152ms, sequenceid=212, compaction requested=false at 1697385873495 (+7 ms)Writing region close event to WAL at 1697385873500 (+5 ms)Closed at 1697385873500 2023-10-15T16:04:33,507 INFO [M:1;8c5d72df62de:41825 {}] flush.MasterFlushTableProcedureManager(85): stop: server shutting down. 2023-10-15T16:04:33,507 INFO [master:store-WAL-Roller {}] wal.AbstractWALRoller(249): LogRoller exiting. 2023-10-15T16:04:33,507 INFO [M:1;8c5d72df62de:41825 {}] ipc.NettyRpcServer(340): Stopping server on /172.17.0.3:41825 2023-10-15T16:04:33,508 INFO [M:1;8c5d72df62de:41825 {}] hbase.HBaseServerBase(476): Close zookeeper 2023-10-15T16:04:33,609 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:41815-0x101b6f97f510004, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:33,609 INFO [RS:2;8c5d72df62de:41815 {}] hbase.HBaseServerBase(483): Close table descriptors 2023-10-15T16:04:33,609 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): regionserver:41815-0x101b6f97f510004, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:33,609 INFO [RS:2;8c5d72df62de:41815 {}] regionserver.HRegionServer(1015): Exiting; stopping=8c5d72df62de,41815,1697385851922; zookeeper connection closed. 2023-10-15T16:04:33,609 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@297ed5a8 {}] hbase.SingleProcessHBaseCluster$SingleFileSystemShutdownThread(211): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@297ed5a8 2023-10-15T16:04:33,610 INFO [Listener at localhost/35085 {}] util.JVMClusterUtil(335): Shutdown of 2 master(s) and 3 regionserver(s) complete 2023-10-15T16:04:33,809 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:33,809 INFO [M:1;8c5d72df62de:41825 {}] hbase.HBaseServerBase(483): Close table descriptors 2023-10-15T16:04:33,809 DEBUG [Listener at localhost/38557-EventThread {}] zookeeper.ZKWatcher(604): master:41825-0x101b6f97f510001, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:33,811 WARN [Listener at localhost/35085 {}] datanode.DirectoryScanner(344): DirectoryScanner: shutdown has been called 2023-10-15T16:04:33,815 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.w.WebAppContext@3dfafe49{datanode,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-m2/hbase-master-patch-1/org/apache/hadoop/hadoop-hdfs/3.2.4/hadoop-hdfs-3.2.4-tests.jar!/webapps/datanode} 2023-10-15T16:04:33,818 INFO [Listener at localhost/35085 {}] server.AbstractConnector(381): Stopped ServerConnector@4df71ee2{HTTP/1.1, (http/1.1)}{localhost:0} 2023-10-15T16:04:33,818 INFO [Listener at localhost/35085 {}] session.HouseKeeper(149): node0 Stopped scavenging 2023-10-15T16:04:33,818 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.s.ServletContextHandler@3c287a65{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-m2/hbase-master-patch-1/org/apache/hadoop/hadoop-hdfs/3.2.4/hadoop-hdfs-3.2.4-tests.jar!/webapps/static,STOPPED} 2023-10-15T16:04:33,818 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.s.ServletContextHandler@57596f4f{logs,/logs,file:///home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/hadoop.log.dir/,STOPPED} 2023-10-15T16:04:33,822 WARN [BP-1681260002-172.17.0.3-1697385845057 heartbeating to localhost/127.0.0.1:40215 {}] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-10-15T16:04:33,822 WARN [BP-1681260002-172.17.0.3-1697385845057 heartbeating to localhost/127.0.0.1:40215 {}] datanode.BPServiceActor(866): Ending block pool service for: Block pool BP-1681260002-172.17.0.3-1697385845057 (Datanode Uuid 7eb54494-67a8-4f0e-b9cc-4c8bfd2a6ecb) service to localhost/127.0.0.1:40215 2023-10-15T16:04:33,822 ERROR [Command processor {}] datanode.BPServiceActor$CommandProcessingThread(1296): Command processor encountered interrupt and exit. 2023-10-15T16:04:33,822 WARN [Command processor {}] datanode.BPServiceActor$CommandProcessingThread(1280): Ending command processor service for: Thread[Command processor,5,FailOnTimeoutGroup] 2023-10-15T16:04:33,823 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data5/current/BP-1681260002-172.17.0.3-1697385845057 {}] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-10-15T16:04:33,823 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data6/current/BP-1681260002-172.17.0.3-1697385845057 {}] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-10-15T16:04:33,825 WARN [Listener at localhost/35085 {}] datanode.DirectoryScanner(344): DirectoryScanner: shutdown has been called 2023-10-15T16:04:33,827 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.w.WebAppContext@294c2ee2{datanode,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-m2/hbase-master-patch-1/org/apache/hadoop/hadoop-hdfs/3.2.4/hadoop-hdfs-3.2.4-tests.jar!/webapps/datanode} 2023-10-15T16:04:33,828 INFO [Listener at localhost/35085 {}] server.AbstractConnector(381): Stopped ServerConnector@6bf03fc9{HTTP/1.1, (http/1.1)}{localhost:0} 2023-10-15T16:04:33,828 INFO [Listener at localhost/35085 {}] session.HouseKeeper(149): node0 Stopped scavenging 2023-10-15T16:04:33,828 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.s.ServletContextHandler@e9934cf{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-m2/hbase-master-patch-1/org/apache/hadoop/hadoop-hdfs/3.2.4/hadoop-hdfs-3.2.4-tests.jar!/webapps/static,STOPPED} 2023-10-15T16:04:33,828 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.s.ServletContextHandler@cde4f16{logs,/logs,file:///home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/hadoop.log.dir/,STOPPED} 2023-10-15T16:04:33,832 ERROR [Command processor {}] datanode.BPServiceActor$CommandProcessingThread(1296): Command processor encountered interrupt and exit. 2023-10-15T16:04:33,832 WARN [Command processor {}] datanode.BPServiceActor$CommandProcessingThread(1280): Ending command processor service for: Thread[Command processor,5,FailOnTimeoutGroup] 2023-10-15T16:04:33,832 WARN [BP-1681260002-172.17.0.3-1697385845057 heartbeating to localhost/127.0.0.1:40215 {}] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-10-15T16:04:33,832 WARN [BP-1681260002-172.17.0.3-1697385845057 heartbeating to localhost/127.0.0.1:40215 {}] datanode.BPServiceActor(866): Ending block pool service for: Block pool BP-1681260002-172.17.0.3-1697385845057 (Datanode Uuid 15c66236-f249-454a-b7a4-7c0458602591) service to localhost/127.0.0.1:40215 2023-10-15T16:04:33,832 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data3/current/BP-1681260002-172.17.0.3-1697385845057 {}] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-10-15T16:04:33,833 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data4/current/BP-1681260002-172.17.0.3-1697385845057 {}] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-10-15T16:04:33,834 WARN [Listener at localhost/35085 {}] datanode.DirectoryScanner(344): DirectoryScanner: shutdown has been called 2023-10-15T16:04:33,837 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.w.WebAppContext@709f20f5{datanode,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-m2/hbase-master-patch-1/org/apache/hadoop/hadoop-hdfs/3.2.4/hadoop-hdfs-3.2.4-tests.jar!/webapps/datanode} 2023-10-15T16:04:33,837 INFO [Listener at localhost/35085 {}] server.AbstractConnector(381): Stopped ServerConnector@5cf8cd9b{HTTP/1.1, (http/1.1)}{localhost:0} 2023-10-15T16:04:33,837 INFO [Listener at localhost/35085 {}] session.HouseKeeper(149): node0 Stopped scavenging 2023-10-15T16:04:33,837 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.s.ServletContextHandler@79ce0709{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-m2/hbase-master-patch-1/org/apache/hadoop/hadoop-hdfs/3.2.4/hadoop-hdfs-3.2.4-tests.jar!/webapps/static,STOPPED} 2023-10-15T16:04:33,837 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.s.ServletContextHandler@43fe3d56{logs,/logs,file:///home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/hadoop.log.dir/,STOPPED} 2023-10-15T16:04:33,841 WARN [BP-1681260002-172.17.0.3-1697385845057 heartbeating to localhost/127.0.0.1:40215 {}] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-10-15T16:04:33,841 ERROR [Command processor {}] datanode.BPServiceActor$CommandProcessingThread(1296): Command processor encountered interrupt and exit. 2023-10-15T16:04:33,841 WARN [BP-1681260002-172.17.0.3-1697385845057 heartbeating to localhost/127.0.0.1:40215 {}] datanode.BPServiceActor(866): Ending block pool service for: Block pool BP-1681260002-172.17.0.3-1697385845057 (Datanode Uuid 383e1eac-8226-4c7d-823e-805983ccff27) service to localhost/127.0.0.1:40215 2023-10-15T16:04:33,841 WARN [Command processor {}] datanode.BPServiceActor$CommandProcessingThread(1280): Ending command processor service for: Thread[Command processor,5,FailOnTimeoutGroup] 2023-10-15T16:04:33,841 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data1/current/BP-1681260002-172.17.0.3-1697385845057 {}] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-10-15T16:04:33,841 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data2/current/BP-1681260002-172.17.0.3-1697385845057 {}] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-10-15T16:04:33,850 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.w.WebAppContext@78f3498c{hdfs,/,null,STOPPED}{jar:file:/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-m2/hbase-master-patch-1/org/apache/hadoop/hadoop-hdfs/3.2.4/hadoop-hdfs-3.2.4-tests.jar!/webapps/hdfs} 2023-10-15T16:04:33,851 INFO [Listener at localhost/35085 {}] server.AbstractConnector(381): Stopped ServerConnector@c90cac1{HTTP/1.1, (http/1.1)}{localhost:0} 2023-10-15T16:04:33,851 INFO [Listener at localhost/35085 {}] session.HouseKeeper(149): node0 Stopped scavenging 2023-10-15T16:04:33,852 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.s.ServletContextHandler@7d6e4756{static,/static,jar:file:/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-m2/hbase-master-patch-1/org/apache/hadoop/hadoop-hdfs/3.2.4/hadoop-hdfs-3.2.4-tests.jar!/webapps/static,STOPPED} 2023-10-15T16:04:33,852 INFO [Listener at localhost/35085 {}] handler.ContextHandler(1153): Stopped o.e.j.s.ServletContextHandler@187fd442{logs,/logs,file:///home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/hadoop.log.dir/,STOPPED} 2023-10-15T16:04:33,894 INFO [Listener at localhost/35085 {}] hbase.HBaseTestingUtil(1048): Minicluster is down 2023-10-15T16:04:33,895 DEBUG [Listener at localhost/35085 {}] replication.TestSyncReplicationStandbyKillRS(100): Going to transit peer 1 to DOWNGRADE_ACTIVE state 2023-10-15T16:04:33,895 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(75): Going to stop 3 RSes: [8c5d72df62de,38435,1697385859175, 8c5d72df62de,46527,1697385859222, 8c5d72df62de,45999,1697385859285] 2023-10-15T16:04:33,895 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(80): Going to RS stop [8c5d72df62de,38435,1697385859175] 2023-10-15T16:04:33,895 INFO [RegionServer-Restarter {}] regionserver.HRegionServer(2147): ***** STOPPING region server '8c5d72df62de,38435,1697385859175' ***** 2023-10-15T16:04:33,895 INFO [RegionServer-Restarter {}] regionserver.HRegionServer(2161): STOPPED: Stop RS for test 2023-10-15T16:04:33,895 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(131): Waiting for 8c5d72df62de,38435,1697385859175 to be listed as dead in master 2023-10-15T16:04:33,895 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.HMaster(3935): Client=jenkins//172.17.0.3 transit current cluster state to DOWNGRADE_ACTIVE in a synchronous replication peer id=1 2023-10-15T16:04:33,895 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.HeapMemoryManager(220): Stopping 2023-10-15T16:04:33,896 INFO [MemStoreFlusher.0 {}] regionserver.MemStoreFlusher$FlushHandler(374): MemStoreFlusher.0 exiting 2023-10-15T16:04:33,896 INFO [RS:0;8c5d72df62de:38435 {}] flush.RegionServerFlushTableProcedureManager(119): Stopping region server flush procedure manager gracefully. 2023-10-15T16:04:33,896 INFO [RS:0;8c5d72df62de:38435 {}] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-10-15T16:04:33,896 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.HRegionServer(3035): Received CLOSE for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:33,896 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.HRegionServer(943): stopping server 8c5d72df62de,38435,1697385859175 2023-10-15T16:04:33,896 INFO [RS:0;8c5d72df62de:38435 {}] hbase.HBaseServerBase(452): Close async cluster connection 2023-10-15T16:04:33,896 INFO [RS:0;8c5d72df62de:38435 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:0;8c5d72df62de:38435. 2023-10-15T16:04:33,897 DEBUG [RS:0;8c5d72df62de:38435 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.HBaseServerBase.closeClusterConnection(HBaseServerBase.java:454) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:946) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:33,897 DEBUG [RS:0;8c5d72df62de:38435 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:33,897 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1710): Closing 2d525cde81451d9cc8ed5e98268bcae5, disabling compactions & flushes 2023-10-15T16:04:33,897 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1743): Closing region hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:33,897 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.HRegionServer(1296): Waiting on 1 regions to close 2023-10-15T16:04:33,897 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1764): Time limited wait for close lock on hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:33,897 DEBUG [RS:0;8c5d72df62de:38435 {}] regionserver.HRegionServer(1300): Online Regions={2d525cde81451d9cc8ed5e98268bcae5=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.} 2023-10-15T16:04:33,897 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1831): Acquired close lock on hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. after waiting 0 ms 2023-10-15T16:04:33,897 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1841): Updates disabled for region hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:33,897 DEBUG [RS:0;8c5d72df62de:38435 {}] regionserver.HRegionServer(1326): Waiting on 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:33,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] procedure2.ProcedureExecutor(1032): Stored pid=24, state=RUNNABLE:PRE_PEER_SYNC_REPLICATION_STATE_TRANSITION, hasLock=false; org.apache.hadoop.hbase.master.replication.TransitPeerSyncReplicationStateProcedure 2023-10-15T16:04:33,897 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(2885): Flushing 2d525cde81451d9cc8ed5e98268bcae5 3/3 column families, dataSize=1.53 KB heapSize=3.48 KB 2023-10-15T16:04:33,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=24 2023-10-15T16:04:33,903 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=26, ppid=24, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=27, ppid=24, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-10-15T16:04:33,921 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/hfileref/06c1db2ad5514d38945c79bef8c56cc7 is 66, key is 1-8c5d72df62de,38435,1697385859175/hfileref:/1697385866877/DeleteFamily/seqid=0 2023-10-15T16:04:33,935 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=186 B at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/hfileref/06c1db2ad5514d38945c79bef8c56cc7 2023-10-15T16:04:33,943 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 06c1db2ad5514d38945c79bef8c56cc7 2023-10-15T16:04:33,963 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/queue/46fdfd784a1242dd9339a7a4fc97d13a is 152, key is 1-8c5d72df62de,38435,1697385859175/queue:8c5d72df62de%2C38435%2C1697385859175/1697385867421/Put/seqid=0 2023-10-15T16:04:33,971 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] hbase.ScheduledChore(149): Chore: CompactionChecker was stopped 2023-10-15T16:04:33,971 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] hbase.ScheduledChore(149): Chore: MemstoreFlusherChore was stopped 2023-10-15T16:04:33,972 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=1.18 KB at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/queue/46fdfd784a1242dd9339a7a4fc97d13a 2023-10-15T16:04:33,978 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46fdfd784a1242dd9339a7a4fc97d13a 2023-10-15T16:04:33,983 INFO [regionserver/8c5d72df62de:0.leaseChecker {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:33,995 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/sid/748fb97f697941489dd7ef4042a092d8 is 61, key is 1-8c5d72df62de,38435,1697385859175/sid:/1697385866877/DeleteFamily/seqid=0 2023-10-15T16:04:34,010 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=171 B at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/sid/748fb97f697941489dd7ef4042a092d8 2023-10-15T16:04:34,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=24 2023-10-15T16:04:34,016 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 748fb97f697941489dd7ef4042a092d8 2023-10-15T16:04:34,017 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/hfileref/06c1db2ad5514d38945c79bef8c56cc7 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/hfileref/06c1db2ad5514d38945c79bef8c56cc7 2023-10-15T16:04:34,022 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 06c1db2ad5514d38945c79bef8c56cc7 2023-10-15T16:04:34,023 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/hfileref/06c1db2ad5514d38945c79bef8c56cc7, entries=3, sequenceid=14, filesize=5.3 K 2023-10-15T16:04:34,024 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/queue/46fdfd784a1242dd9339a7a4fc97d13a as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/46fdfd784a1242dd9339a7a4fc97d13a 2023-10-15T16:04:34,029 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46fdfd784a1242dd9339a7a4fc97d13a 2023-10-15T16:04:34,029 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/46fdfd784a1242dd9339a7a4fc97d13a, entries=6, sequenceid=14, filesize=5.8 K 2023-10-15T16:04:34,030 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/sid/748fb97f697941489dd7ef4042a092d8 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/sid/748fb97f697941489dd7ef4042a092d8 2023-10-15T16:04:34,036 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 748fb97f697941489dd7ef4042a092d8 2023-10-15T16:04:34,036 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/sid/748fb97f697941489dd7ef4042a092d8, entries=3, sequenceid=14, filesize=5.3 K 2023-10-15T16:04:34,037 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(3123): Finished flush of dataSize ~1.53 KB/1568, heapSize ~3.43 KB/3512, currentSize=0 B/0 for 2d525cde81451d9cc8ed5e98268bcae5 in 140ms, sequenceid=14, compaction requested=false 2023-10-15T16:04:34,046 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] wal.WALSplitUtil(410): Wrote file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-10-15T16:04:34,046 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-15T16:04:34,047 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1961): Closed hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:34,047 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1664): Region close journal for 2d525cde81451d9cc8ed5e98268bcae5: Waiting for close lock at 1697385873896Running coprocessor pre-close hooks at 1697385873896Disabling compacts and flushes for region at 1697385873896Disabling writes for close at 1697385873897 (+1 ms)Obtaining lock to block concurrent updates at 1697385873897Preparing flush snapshotting stores in 2d525cde81451d9cc8ed5e98268bcae5 at 1697385873897Finished memstore snapshotting hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., syncing WAL and waiting on mvcc, flushsize=dataSize=1568, getHeapSize=3512, getOffHeapSize=0, getCellsCount=16 at 1697385873898 (+1 ms)Flushing stores of hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. at 1697385873899 (+1 ms)Flushing 2d525cde81451d9cc8ed5e98268bcae5/hfileref: creating writer at 1697385873899Flushing 2d525cde81451d9cc8ed5e98268bcae5/hfileref: appending metadata at 1697385873919 (+20 ms)Flushing 2d525cde81451d9cc8ed5e98268bcae5/hfileref: closing flushed file at 1697385873919Flushing 2d525cde81451d9cc8ed5e98268bcae5/queue: creating writer at 1697385873943 (+24 ms)Flushing 2d525cde81451d9cc8ed5e98268bcae5/queue: appending metadata at 1697385873962 (+19 ms)Flushing 2d525cde81451d9cc8ed5e98268bcae5/queue: closing flushed file at 1697385873962Flushing 2d525cde81451d9cc8ed5e98268bcae5/sid: creating writer at 1697385873978 (+16 ms)Flushing 2d525cde81451d9cc8ed5e98268bcae5/sid: appending metadata at 1697385873994 (+16 ms)Flushing 2d525cde81451d9cc8ed5e98268bcae5/sid: closing flushed file at 1697385873995 (+1 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@1766dfe2: reopening flushed file at 1697385874016 (+21 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@3053a736: reopening flushed file at 1697385874023 (+7 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@7677cbca: reopening flushed file at 1697385874029 (+6 ms)Finished flush of dataSize ~1.53 KB/1568, heapSize ~3.43 KB/3512, currentSize=0 B/0 for 2d525cde81451d9cc8ed5e98268bcae5 in 140ms, sequenceid=14, compaction requested=false at 1697385874037 (+8 ms)Writing region close event to WAL at 1697385874043 (+6 ms)Running coprocessor post-close hooks at 1697385874046 (+3 ms)Closed at 1697385874047 (+1 ms) 2023-10-15T16:04:34,047 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] handler.CloseRegionHandler(117): Closed hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:34,056 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46527 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=27 2023-10-15T16:04:34,056 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45999 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=26 2023-10-15T16:04:34,056 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:34,056 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:34,056 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38435 {}] ipc.MetricsHBaseServer(152): Unknown exception type org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server 8c5d72df62de,38435,1697385859175 stopping at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1426) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.RSRpcServices.executeProcedures(RSRpcServices.java:3891) ~[classes/:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:34783) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437) ~[classes/:?] at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124) ~[classes/:?] at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102) ~[classes/:?] at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) ~[classes/:?] 2023-10-15T16:04:34,057 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38435 {}] ipc.CallRunner(138): callId: 40 service: AdminService methodName: ExecuteProcedures size: 143 connection: 172.17.0.3:51305 deadline: 1697385934056, exception=org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server 8c5d72df62de,38435,1697385859175 stopping 2023-10-15T16:04:34,057 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=27}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=27 2023-10-15T16:04:34,057 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=26}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=26 2023-10-15T16:04:34,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=26 2023-10-15T16:04:34,058 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,45999,1697385859285 suceeded 2023-10-15T16:04:34,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=27 2023-10-15T16:04:34,059 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,46527,1697385859222 suceeded 2023-10-15T16:04:34,058 DEBUG [RSProcedureDispatcher-pool-0 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(271): Request to 8c5d72df62de,38435,1697385859175 failed, try=0 org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server 8c5d72df62de,38435,1697385859175 stopping at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1426) at org.apache.hadoop.hbase.regionserver.RSRpcServices.executeProcedures(RSRpcServices.java:3891) at org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:34783) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124) at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102) at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:110) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:100) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.unwrapException(RSProcedureDispatcher.java:384) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:400) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server 8c5d72df62de,38435,1697385859175 stopping at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1426) at org.apache.hadoop.hbase.regionserver.RSRpcServices.executeProcedures(RSRpcServices.java:3891) at org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:34783) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124) at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102) at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:186) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.sendRequest(RSProcedureDispatcher.java:431) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:398) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:391) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:197) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:218) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:04:34,061 WARN [RSProcedureDispatcher-pool-0 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(314): 8c5d72df62de,38435,1697385859175 is aborted or stopped, for safety we still need to wait until it is fully dead, try=0 2023-10-15T16:04:34,062 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=26, ppid=24, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 158 msec 2023-10-15T16:04:34,062 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1413): Finished pid=27, ppid=24, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 158 msec 2023-10-15T16:04:34,097 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.HRegionServer(960): stopping server 8c5d72df62de,38435,1697385859175; all regions closed. 2023-10-15T16:04:34,104 DEBUG [RS:0;8c5d72df62de:38435 {}] wal.AbstractFSWAL(1244): Moved 2 WAL file(s) to /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs 2023-10-15T16:04:34,104 INFO [RS:0;8c5d72df62de:38435 {}] wal.AbstractFSWAL(1247): Closed WAL: AsyncFSWAL 8c5d72df62de%2C38435%2C1697385859175.rep:(num 1697385867425) 2023-10-15T16:04:34,109 DEBUG [RS:0;8c5d72df62de:38435 {}] wal.AbstractFSWAL(1244): Moved 1 WAL file(s) to /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs 2023-10-15T16:04:34,109 INFO [RS:0;8c5d72df62de:38435 {}] wal.AbstractFSWAL(1247): Closed WAL: AsyncFSWAL 8c5d72df62de%2C38435%2C1697385859175:(num 1697385867465) 2023-10-15T16:04:34,109 DEBUG [RS:0;8c5d72df62de:38435 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:34,109 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:34,109 INFO [RS:0;8c5d72df62de:38435 {}] hbase.HBaseServerBase(435): Shutdown chores and chore service 2023-10-15T16:04:34,109 INFO [RS:0;8c5d72df62de:38435 {}] hbase.ChoreService(370): Chore service for: regionserver/8c5d72df62de:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=ReplicationSourceStatistics, period=5000, unit=MILLISECONDS, ScheduledChore name=ReplicationSinkStatistics, period=5000, unit=MILLISECONDS] on shutdown 2023-10-15T16:04:34,110 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.CompactSplit(469): Waiting for Split Thread to finish... 2023-10-15T16:04:34,110 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.CompactSplit(469): Waiting for Large Compaction Thread to finish... 2023-10-15T16:04:34,110 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.CompactSplit(469): Waiting for Small Compaction Thread to finish... 2023-10-15T16:04:34,110 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(249): LogRoller exiting. 2023-10-15T16:04:34,110 INFO [RS:0;8c5d72df62de:38435 {}] hbase.HBaseServerBase(445): Shutdown executor service 2023-10-15T16:04:34,110 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,38435,1697385859175 because: Region server is closing 2023-10-15T16:04:34,162 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38435 {}] ipc.MetricsHBaseServer(152): Unknown exception type org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server 8c5d72df62de,38435,1697385859175 stopping at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1426) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.RSRpcServices.executeProcedures(RSRpcServices.java:3891) ~[classes/:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:34783) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437) ~[classes/:?] at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124) ~[classes/:?] at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102) ~[classes/:?] at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) ~[classes/:?] 2023-10-15T16:04:34,162 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38435 {}] ipc.CallRunner(138): callId: 43 service: AdminService methodName: ExecuteProcedures size: 143 connection: 172.17.0.3:51305 deadline: 1697385934162, exception=org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server 8c5d72df62de,38435,1697385859175 stopping 2023-10-15T16:04:34,164 DEBUG [RSProcedureDispatcher-pool-2 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(271): Request to 8c5d72df62de,38435,1697385859175 failed, try=1 org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server 8c5d72df62de,38435,1697385859175 stopping at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1426) at org.apache.hadoop.hbase.regionserver.RSRpcServices.executeProcedures(RSRpcServices.java:3891) at org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:34783) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124) at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102) at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:110) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:100) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.unwrapException(RSProcedureDispatcher.java:384) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:400) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server 8c5d72df62de,38435,1697385859175 stopping at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1426) at org.apache.hadoop.hbase.regionserver.RSRpcServices.executeProcedures(RSRpcServices.java:3891) at org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:34783) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:437) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124) at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:102) at org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:186) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.sendRequest(RSProcedureDispatcher.java:431) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:398) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:391) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:197) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:218) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:04:34,165 WARN [RSProcedureDispatcher-pool-2 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(314): 8c5d72df62de,38435,1697385859175 is aborted or stopped, for safety we still need to wait until it is fully dead, try=1 2023-10-15T16:04:34,213 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,38435,1697385859175-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f510022, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:34,213 INFO [RS:0;8c5d72df62de:38435 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:0;8c5d72df62de:38435. 2023-10-15T16:04:34,214 DEBUG [RS:0;8c5d72df62de:38435 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:664) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:659) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:987) at org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:160) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2481) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:995) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:34,214 DEBUG [RS:0;8c5d72df62de:38435 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:34,214 DEBUG [RS:0;8c5d72df62de:38435 {}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x2ffdd842 to 127.0.0.1:52753 2023-10-15T16:04:34,218 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 0, reset compression=false 2023-10-15T16:04:34,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=24 2023-10-15T16:04:34,223 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:34,299 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:34,304 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:34,314 WARN [regionserver/8c5d72df62de:0.logRoller.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) ~[?:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:307) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) ~[classes/:?] 2023-10-15T16:04:34,315 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker regionserver/8c5d72df62de:0.logRoller.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,38435,1697385859175 terminated 2023-10-15T16:04:34,315 INFO [RS:0;8c5d72df62de:38435 {}] ipc.NettyRpcServer(340): Stopping server on /172.17.0.3:38435 2023-10-15T16:04:34,320 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): regionserver:38435-0x101b6f97f510007, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster2/rs/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:34,320 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster2/rs 2023-10-15T16:04:34,320 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:38213-0x101b6f97f510005, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster2/rs 2023-10-15T16:04:34,320 INFO [RS:0;8c5d72df62de:38435 {}] hbase.HBaseServerBase(476): Close zookeeper 2023-10-15T16:04:34,321 INFO [RegionServerTracker-0 {}] master.RegionServerTracker(171): RegionServer ephemeral node deleted, processing expiration [8c5d72df62de,38435,1697385859175] 2023-10-15T16:04:34,324 DEBUG [RegionServerTracker-0 {}] zookeeper.RecoverableZooKeeper(221): Node /cluster2/draining/8c5d72df62de,38435,1697385859175 already deleted, retry=false 2023-10-15T16:04:34,324 INFO [RegionServerTracker-0 {}] master.ServerManager(604): Processing expiration of 8c5d72df62de,38435,1697385859175 on 8c5d72df62de,33403,1697385859093 2023-10-15T16:04:34,330 DEBUG [RegionServerTracker-0 {}] procedure2.ProcedureExecutor(1032): Stored pid=28, state=RUNNABLE:SERVER_CRASH_START, hasLock=false; ServerCrashProcedure 8c5d72df62de,38435,1697385859175, splitWal=true, meta=false 2023-10-15T16:04:34,330 INFO [RegionServerTracker-0 {}] assignment.AssignmentManager(1862): Scheduled ServerCrashProcedure pid=28 for 8c5d72df62de,38435,1697385859175 (carryingMeta=false) 8c5d72df62de,38435,1697385859175/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@17fa99f[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-10-15T16:04:34,333 INFO [PEWorker-4 {}] procedure.ServerCrashProcedure(163): Start pid=28, state=RUNNABLE:SERVER_CRASH_START, hasLock=true; ServerCrashProcedure 8c5d72df62de,38435,1697385859175, splitWal=true, meta=false 2023-10-15T16:04:34,332 WARN [RegionServerTracker-0 {}] replication.RefreshPeerProcedure(126): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,38435,1697385859175 failed org.apache.hadoop.hbase.DoNotRetryIOException: server not online 8c5d72df62de,38435,1697385859175 at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher.abortPendingOperations(RSProcedureDispatcher.java:163) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher.abortPendingOperations(RSProcedureDispatcher.java:61) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher$BufferNode.abortOperationsInQueue(RemoteProcedureDispatcher.java:417) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.removeNode(RemoteProcedureDispatcher.java:201) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher.serverRemoved(RSProcedureDispatcher.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.ServerManager.lambda$expireServer$3(ServerManager.java:609) ~[classes/:?] at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) ~[?:?] at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[?:?] at org.apache.hadoop.hbase.master.ServerManager.expireServer(ServerManager.java:609) ~[classes/:?] at org.apache.hadoop.hbase.master.ServerManager.expireServer(ServerManager.java:563) ~[classes/:?] at org.apache.hadoop.hbase.master.RegionServerTracker.processAsActiveMaster(RegionServerTracker.java:172) ~[classes/:?] at org.apache.hadoop.hbase.master.RegionServerTracker.refresh(RegionServerTracker.java:206) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:34,337 WARN [PEWorker-3 {}] procedure.ServerRemoteProcedure(98): Can not send remote operation 25 to 8c5d72df62de,38435,1697385859175, this operation will be retried to send to another server 2023-10-15T16:04:34,337 INFO [PEWorker-4 {}] procedure.ServerCrashProcedure(201): 8c5d72df62de,38435,1697385859175 had 1 regions 2023-10-15T16:04:34,339 INFO [PEWorker-4 {}] procedure.ServerCrashProcedure(333): Splitting WALs pid=28, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, hasLock=true; ServerCrashProcedure 8c5d72df62de,38435,1697385859175, splitWal=true, meta=false, isMeta: false 2023-10-15T16:04:34,340 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=25, resume processing ppid=24 2023-10-15T16:04:34,340 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=25, ppid=24, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 434 msec 2023-10-15T16:04:34,340 WARN [PEWorker-3 {}] procedure2.RemoteProcedureDispatcher(184): since no node for this key 8c5d72df62de,38435,1697385859175, we can't removed the finished remote procedure 2023-10-15T16:04:34,341 DEBUG [PEWorker-4 {}] master.MasterWalManager(317): Renamed region directory: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175-splitting 2023-10-15T16:04:34,342 INFO [PEWorker-4 {}] master.SplitLogManager(171): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175-splitting dir is empty, no logs to split. 2023-10-15T16:04:34,342 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=29, ppid=24, state=RUNNABLE:RENAME_SYNC_REPLICATION_WALS_DIR, hasLock=false; org.apache.hadoop.hbase.master.replication.RecoverStandbyProcedure}] 2023-10-15T16:04:34,342 INFO [PEWorker-4 {}] master.SplitWALManager(105): 8c5d72df62de,38435,1697385859175 WAL count=0, meta=false 2023-10-15T16:04:34,347 INFO [PEWorker-4 {}] master.SplitLogManager(171): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175-splitting dir is empty, no logs to split. 2023-10-15T16:04:34,347 INFO [PEWorker-4 {}] master.SplitWALManager(105): 8c5d72df62de,38435,1697385859175 WAL count=0, meta=false 2023-10-15T16:04:34,347 DEBUG [PEWorker-4 {}] procedure.ServerCrashProcedure(323): Check if 8c5d72df62de,38435,1697385859175 WAL splitting is done? wals=0, meta=false 2023-10-15T16:04:34,348 INFO [PEWorker-1 {}] replication.SyncReplicationReplayWALManager(221): Renamed dir from hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/remoteWALs/1 to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/remoteWALs/1-replay for peer id=1 2023-10-15T16:04:34,349 INFO [PEWorker-4 {}] procedure.ServerCrashProcedure(315): Remove WAL directory for 8c5d72df62de,38435,1697385859175 failed, ignore...File hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,38435,1697385859175-splitting does not exist. 2023-10-15T16:04:34,350 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=30, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN}] 2023-10-15T16:04:34,352 INFO [PEWorker-5 {}] procedure.MasterProcedureScheduler(792): Took xlock for pid=30, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN 2023-10-15T16:04:34,353 INFO [PEWorker-5 {}] assignment.TransitRegionStateProcedure(262): Starting pid=30, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-10-15T16:04:34,354 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=31, ppid=29, state=RUNNABLE:ASSIGN_WORKER, hasLock=false; org.apache.hadoop.hbase.master.replication.SyncReplicationReplayWALProcedure}] 2023-10-15T16:04:34,360 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=32, ppid=31, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.SyncReplicationReplayWALRemoteProcedure}] 2023-10-15T16:04:34,422 INFO [RS:0;8c5d72df62de:38435 {}] hbase.HBaseServerBase(483): Close table descriptors 2023-10-15T16:04:34,422 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): regionserver:38435-0x101b6f97f510007, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:34,422 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): regionserver:38435-0x101b6f97f510007, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:34,422 INFO [RS:0;8c5d72df62de:38435 {}] regionserver.HRegionServer(1015): Exiting; stopping=8c5d72df62de,38435,1697385859175; zookeeper connection closed. 2023-10-15T16:04:34,422 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@25a00aa0 {}] hbase.SingleProcessHBaseCluster$SingleFileSystemShutdownThread(211): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@25a00aa0 2023-10-15T16:04:34,503 DEBUG [8c5d72df62de:33403 {}] assignment.AssignmentManager(2316): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-10-15T16:04:34,503 DEBUG [8c5d72df62de:33403 {}] balancer.BalancerClusterState(188): Hosts are {8c5d72df62de=0} racks are {/default-rack=0} 2023-10-15T16:04:34,504 DEBUG [8c5d72df62de:33403 {}] balancer.BalancerClusterState(287): server 0 has 0 regions 2023-10-15T16:04:34,504 DEBUG [8c5d72df62de:33403 {}] balancer.BalancerClusterState(287): server 1 has 0 regions 2023-10-15T16:04:34,504 DEBUG [8c5d72df62de:33403 {}] balancer.BalancerClusterState(294): server 0 is on host 0 2023-10-15T16:04:34,504 DEBUG [8c5d72df62de:33403 {}] balancer.BalancerClusterState(294): server 1 is on host 0 2023-10-15T16:04:34,504 INFO [8c5d72df62de:33403 {}] balancer.BalancerClusterState(305): server 0 is on rack 0 2023-10-15T16:04:34,504 INFO [8c5d72df62de:33403 {}] balancer.BalancerClusterState(305): server 1 is on rack 0 2023-10-15T16:04:34,504 DEBUG [8c5d72df62de:33403 {}] balancer.BalancerClusterState(310): Number of tables=1, number of hosts=1, number of racks=1 2023-10-15T16:04:34,504 INFO [PEWorker-4 {}] assignment.RegionStateStore(223): pid=30 updating hbase:meta row=2d525cde81451d9cc8ed5e98268bcae5, regionState=OPENING, regionLocation=8c5d72df62de,45999,1697385859285 2023-10-15T16:04:34,506 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=33, ppid=30, state=RUNNABLE, hasLock=false; OpenRegionProcedure 2d525cde81451d9cc8ed5e98268bcae5, server=8c5d72df62de,45999,1697385859285}] 2023-10-15T16:04:34,522 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] handler.AssignRegionHandler(127): Open hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:34,523 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] wal.WALFactory(196): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-10-15T16:04:34,523 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] monitor.StreamSlowMonitor(122): New stream slow monitor rep 2023-10-15T16:04:34,523 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45999 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.ReplaySyncReplicationWALCallable, pid=32 2023-10-15T16:04:34,524 INFO [RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0-0 {event_type=RS_REPLAY_SYNC_REPLICATION_WAL, pid=32}] regionserver.ReplaySyncReplicationWALCallable(73): Received a replay sync replication wals [remoteWALs/1-replay/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep] event, peerId=1 2023-10-15T16:04:34,524 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] wal.AbstractFSWAL(602): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=8c5d72df62de%2C45999%2C1697385859285.rep, suffix=, logDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285, archiveDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs, maxLogs=10 2023-10-15T16:04:34,525 INFO [RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0-0 {event_type=RS_REPLAY_SYNC_REPLICATION_WAL, pid=32}] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/remoteWALs/1-replay/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep 2023-10-15T16:04:34,530 INFO [RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0-0 {event_type=RS_REPLAY_SYNC_REPLICATION_WAL, pid=32}] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/remoteWALs/1-replay/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep after 5ms 2023-10-15T16:04:34,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=24 2023-10-15T16:04:34,538 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525, exclude list is [], retry=0 2023-10-15T16:04:34,541 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:34,541 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:34,541 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:34,547 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] wal.AbstractFSWAL(980): New WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525 2023-10-15T16:04:34,550 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK], DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK], DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK]] 2023-10-15T16:04:34,551 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(7727): Opening region: {ENCODED => 2d525cde81451d9cc8ed5e98268bcae5, NAME => 'hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.', STARTKEY => '', ENDKEY => ''} 2023-10-15T16:04:34,551 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-10-15T16:04:34,551 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(8255): Registered coprocessor service: region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. service=MultiRowMutationService 2023-10-15T16:04:34,551 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.RegionCoprocessorHost(391): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:replication successfully. 2023-10-15T16:04:34,551 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table replication 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,551 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(891): Instantiated hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=0 ; minColumnNum=100 ; preparePutThreadLimit=0 ; hotProtect now disable 2023-10-15T16:04:34,551 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(7769): checking encryption for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,552 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(7772): checking classloading for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,553 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family hfileref of region 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,554 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d525cde81451d9cc8ed5e98268bcae5 columnFamilyName hfileref 2023-10-15T16:04:34,555 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:34,567 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:38435 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:38435 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:34,568 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:38435 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:38435 2023-10-15T16:04:34,568 DEBUG [RSProcedureDispatcher-pool-0 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(271): Request to 8c5d72df62de,38435,1697385859175 failed, try=2 java.net.ConnectException: Call to address=8c5d72df62de:38435 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:38435 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:186) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.sendRequest(RSProcedureDispatcher.java:431) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:398) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:38435 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:04:34,570 WARN [RSProcedureDispatcher-pool-0 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(305): Request to 8c5d72df62de,38435,1697385859175 failed due to java.net.ConnectException: Call to address=8c5d72df62de:38435 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:38435, try=2 and the server is not online, give up 2023-10-15T16:04:34,570 INFO [RSProcedureDispatcher-pool-0 {}] procedure.ServerRemoteProcedure(132): This procedure 25 is already finished, skip the rest processes 2023-10-15T16:04:34,573 INFO [StoreFileOpener-2d525cde81451d9cc8ed5e98268bcae5-hfileref-1 {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 06c1db2ad5514d38945c79bef8c56cc7 2023-10-15T16:04:34,573 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/hfileref/06c1db2ad5514d38945c79bef8c56cc7 2023-10-15T16:04:34,574 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(324): Store=2d525cde81451d9cc8ed5e98268bcae5/hfileref, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:34,574 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family queue of region 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,575 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d525cde81451d9cc8ed5e98268bcae5 columnFamilyName queue 2023-10-15T16:04:34,575 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:34,583 INFO [StoreFileOpener-2d525cde81451d9cc8ed5e98268bcae5-queue-1 {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46fdfd784a1242dd9339a7a4fc97d13a 2023-10-15T16:04:34,583 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/46fdfd784a1242dd9339a7a4fc97d13a 2023-10-15T16:04:34,583 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(324): Store=2d525cde81451d9cc8ed5e98268bcae5/queue, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:34,583 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family sid of region 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,584 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d525cde81451d9cc8ed5e98268bcae5 columnFamilyName sid 2023-10-15T16:04:34,584 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:34,585 DEBUG [RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0-0 {event_type=RS_REPLAY_SYNC_REPLICATION_WAL, pid=32}] regionserver.ReplicationSink(297): Started replicating mutations. 2023-10-15T16:04:34,585 DEBUG [RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0-0 {event_type=RS_REPLAY_SYNC_REPLICATION_WAL, pid=32}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@1a229bca, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:34,587 DEBUG [RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0-0 {event_type=RS_REPLAY_SYNC_REPLICATION_WAL, pid=32}] ipc.RpcConnection(127): Using SIMPLE authentication for service=ClientMetaService, sasl=false 2023-10-15T16:04:34,588 DEBUG [RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0-0 {event_type=RS_REPLAY_SYNC_REPLICATION_WAL, pid=32}] ipc.RpcConnection(127): Using SIMPLE authentication for service=ClientMetaService, sasl=false 2023-10-15T16:04:34,590 INFO [HMaster-EventLoopGroup-9-1 {}] ipc.ServerRpcConnection(478): Connection from 172.17.0.3:52398, version=4.0.0-alpha-1-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientMetaService 2023-10-15T16:04:34,591 DEBUG [RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0-0 {event_type=RS_REPLAY_SYNC_REPLICATION_WAL, pid=32}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@2545e18c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:34,591 INFO [HMaster-EventLoopGroup-8-3 {}] ipc.ServerRpcConnection(478): Connection from 172.17.0.3:58080, version=4.0.0-alpha-1-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientMetaService 2023-10-15T16:04:34,594 INFO [StoreFileOpener-2d525cde81451d9cc8ed5e98268bcae5-sid-1 {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 748fb97f697941489dd7ef4042a092d8 2023-10-15T16:04:34,594 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/sid/748fb97f697941489dd7ef4042a092d8 2023-10-15T16:04:34,595 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(324): Store=2d525cde81451d9cc8ed5e98268bcae5/sid, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:34,595 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(1028): replaying wal for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,596 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,598 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,599 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(1038): stopping wal replay for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,599 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(1050): Cleaning up temporary data for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,601 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:replication descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-10-15T16:04:34,603 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(1083): writing seq id for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,604 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(1104): Opened 2d525cde81451d9cc8ed5e98268bcae5; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69525316, jitterRate=0.03600794076919556}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-10-15T16:04:34,604 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(1112): Running coprocessor post-open hooks for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:34,604 DEBUG [RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0-0 {event_type=RS_REPLAY_SYNC_REPLICATION_WAL, pid=32}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:04:34,605 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegion(996): Region open journal for 2d525cde81451d9cc8ed5e98268bcae5: Running coprocessor pre-open hook at 1697385874552Writing region info on filesystem at 1697385874552Initializing all the Stores at 1697385874553 (+1 ms)Instantiating store for column family {NAME => 'hfileref', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385874553Instantiating store for column family {NAME => 'queue', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385874553Instantiating store for column family {NAME => 'sid', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385874553Cleaning up temporary data from old regions at 1697385874599 (+46 ms)Running coprocessor post-open hooks at 1697385874604 (+5 ms)Region opened successfully at 1697385874605 (+1 ms) 2023-10-15T16:04:34,606 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,45999,1697385859285, seqNum=-1] 2023-10-15T16:04:34,606 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.RpcConnection(127): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-15T16:04:34,608 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegionServer(2186): Post open deploy tasks for hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., pid=33, masterSystemTime=1697385874518 2023-10-15T16:04:34,612 INFO [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] ipc.ServerRpcConnection(478): Connection from 172.17.0.3:60738, version=4.0.0-alpha-1-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=ClientService 2023-10-15T16:04:34,615 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'SyncRep', row='\x00\x00\x00\x00', locateType=CURRENT is [region=SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1., hostname=8c5d72df62de,45999,1697385859285, seqNum=2] 2023-10-15T16:04:34,616 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] regionserver.HRegionServer(2216): Finished post open deploy task for hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:34,616 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=33}] handler.AssignRegionHandler(147): Opened hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:34,618 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'SyncRep', row='\x00\x00\x00\x01', locateType=CURRENT is [region=SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1., hostname=8c5d72df62de,45999,1697385859285, seqNum=2] 2023-10-15T16:04:34,620 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'SyncRep', row='\x00\x00\x00\x03', locateType=CURRENT is [region=SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1., hostname=8c5d72df62de,45999,1697385859285, seqNum=2] 2023-10-15T16:04:34,621 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'SyncRep', row='\x00\x00\x00\x02', locateType=CURRENT is [region=SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1., hostname=8c5d72df62de,45999,1697385859285, seqNum=2] 2023-10-15T16:04:34,621 INFO [PEWorker-1 {}] assignment.RegionStateStore(223): pid=30 updating hbase:meta row=2d525cde81451d9cc8ed5e98268bcae5, regionState=OPEN, openSeqNum=18, regionLocation=8c5d72df62de,45999,1697385859285 2023-10-15T16:04:34,622 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'SyncRep', row='\x00\x00\x00\x04', locateType=CURRENT is [region=SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1., hostname=8c5d72df62de,45999,1697385859285, seqNum=2] 2023-10-15T16:04:34,623 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'SyncRep', row='\x00\x00\x00\x05', locateType=CURRENT is [region=SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1., hostname=8c5d72df62de,45999,1697385859285, seqNum=2] 2023-10-15T16:04:34,624 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'SyncRep', row='\x00\x00\x00\x06', locateType=CURRENT is [region=SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1., hostname=8c5d72df62de,45999,1697385859285, seqNum=2] 2023-10-15T16:04:34,627 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=33, resume processing ppid=30 2023-10-15T16:04:34,628 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=33, ppid=30, state=SUCCESS, hasLock=false; OpenRegionProcedure 2d525cde81451d9cc8ed5e98268bcae5, server=8c5d72df62de,45999,1697385859285 in 119 msec 2023-10-15T16:04:34,630 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=30, resume processing ppid=28 2023-10-15T16:04:34,630 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=30, ppid=28, state=SUCCESS, hasLock=false; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN in 278 msec 2023-10-15T16:04:34,633 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=34, ppid=28, state=RUNNABLE:ASSIGN_REPLICATION_QUEUES_ADD_MISSING_QUEUES, hasLock=false; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure}] 2023-10-15T16:04:34,641 DEBUG [PEWorker-4 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:38435 this server is in the failed servers list 2023-10-15T16:04:34,643 DEBUG [PEWorker-4 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2 , the old value is region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:38435 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:38435 2023-10-15T16:04:34,644 DEBUG [PEWorker-4 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:38435 2023-10-15T16:04:34,644 DEBUG [PEWorker-4 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2 from cache 2023-10-15T16:04:34,754 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'hbase:replication', row='', locateType=CURRENT is [region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,45999,1697385859285, seqNum=18] 2023-10-15T16:04:34,790 DEBUG [PEWorker-4 {}] replication.AssignReplicationQueuesProcedure(120): There are 1 replication queues need to be claimed for 8c5d72df62de,38435,1697385859175 2023-10-15T16:04:34,791 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=35, ppid=34, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.ClaimReplicationQueueRemoteProcedure}] 2023-10-15T16:04:34,803 DEBUG [RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0-0 {event_type=RS_REPLAY_SYNC_REPLICATION_WAL, pid=32}] regionserver.ReplicationSink(301): Finished replicating mutations. 2023-10-15T16:04:34,804 DEBUG [RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0-0 {event_type=RS_REPLAY_SYNC_REPLICATION_WAL, pid=32}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=32 2023-10-15T16:04:34,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=32 2023-10-15T16:04:34,814 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] replication.SyncReplicationReplayWALRemoteProcedure(80): Replay wals [remoteWALs/1-replay/8c5d72df62de%2C41815%2C1697385851922-1697385868530-1.1697385868532.syncrep] on 8c5d72df62de,45999,1697385859285 succeed for peer id=1 2023-10-15T16:04:34,817 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=32, resume processing ppid=31 2023-10-15T16:04:34,818 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=32, ppid=31, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.SyncReplicationReplayWALRemoteProcedure in 455 msec 2023-10-15T16:04:34,822 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=31, resume processing ppid=29 2023-10-15T16:04:34,822 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=31, ppid=29, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.SyncReplicationReplayWALProcedure in 466 msec 2023-10-15T16:04:34,825 INFO [PEWorker-2 {}] replication.SyncReplicationReplayWALManager(221): Renamed dir from hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/remoteWALs/1-replay to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/remoteWALs/1-snapshot for peer id=1 2023-10-15T16:04:34,827 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=29, resume processing ppid=24 2023-10-15T16:04:34,827 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1413): Finished pid=29, ppid=24, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RecoverStandbyProcedure in 484 msec 2023-10-15T16:04:34,830 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=36, ppid=24, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}, {pid=37, ppid=24, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure}] 2023-10-15T16:04:34,895 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(134): Server 8c5d72df62de,38435,1697385859175 marked as dead, waiting for it to finish dead processing 2023-10-15T16:04:34,896 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:34,968 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45999 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.ClaimReplicationQueueCallable, pid=35 2023-10-15T16:04:34,968 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45999 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=36 2023-10-15T16:04:34,968 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=36}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:34,968 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=36}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=36 2023-10-15T16:04:34,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=36 2023-10-15T16:04:34,969 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,45999,1697385859285 suceeded 2023-10-15T16:04:34,972 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=36, ppid=24, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 141 msec 2023-10-15T16:04:34,981 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46527 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.RefreshPeerCallable, pid=37 2023-10-15T16:04:34,982 INFO [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=37}] regionserver.RefreshPeerCallable(47): Received a peer change event, peerId=1, type=TRANSIT_SYNC_REPLICATION_STATE 2023-10-15T16:04:34,982 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0 {event_type=RS_REFRESH_PEER, pid=37}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=37 2023-10-15T16:04:34,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=37 2023-10-15T16:04:34,982 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] replication.RefreshPeerProcedure(129): Refresh peer 1 for TRANSIT_SYNC_REPLICATION_STATE on 8c5d72df62de,46527,1697385859222 suceeded 2023-10-15T16:04:34,986 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=37, resume processing ppid=24 2023-10-15T16:04:34,986 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1413): Finished pid=37, ppid=24, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.RefreshPeerProcedure in 154 msec 2023-10-15T16:04:34,987 INFO [PEWorker-3 {}] replication.TransitPeerSyncReplicationStateProcedure(135): Successfully transit current cluster state from STANDBY to DOWNGRADE_ACTIVE for sync replication peer 1 2023-10-15T16:04:34,988 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=24, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.TransitPeerSyncReplicationStateProcedure in 1.0920 sec 2023-10-15T16:04:35,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=24 2023-10-15T16:04:35,041 DEBUG [Listener at localhost/35085 {}] replication.TestSyncReplicationStandbyKillRS(110): Waiting for the restarter thread RegionServer-Restarter to quit 2023-10-15T16:04:35,041 INFO [RPCClient-NioEventLoopGroup-7-8 {}] client.RawAsyncHBaseAdmin$ReplicationProcedureBiConsumer(2911): Operation: TRANSIT_REPLICATION_PEER_SYNCHRONOUS_REPLICATION_STATE, peerId: 1 completed 2023-10-15T16:04:35,227 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 0, reset compression=false 2023-10-15T16:04:35,233 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=true 2023-10-15T16:04:35,244 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceWALReader(175): Read 1 WAL entries eligible for replication 2023-10-15T16:04:35,244 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 36149, reset compression=false 2023-10-15T16:04:35,249 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=36149, beingWritten=true 2023-10-15T16:04:35,244 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,45999,1697385859285 got entry batch from reader: WALEntryBatch [walEntries=[{SyncRep/f3bcdec0351888cb7677be57c1f453c1/4=[#edits: 1000 = <\x00\x00\x00\x00/cf:cq/1697385870586/Put/vlen=4/seqid=0; \x00\x00\x00\x01/cf:cq/1697385870592/Put/vlen=4/seqid=0; \x00\x00\x00\x02/cf:cq/1697385870595/Put/vlen=4/seqid=0; \x00\x00\x00\x03/cf:cq/1697385870598/Put/vlen=4/seqid=0; \x00\x00\x00\x04/cf:cq/1697385870600/Put/vlen=4/seqid=0; \x00\x00\x00\x05/cf:cq/1697385870602/Put/vlen=4/seqid=0; \x00\x00\x00\x06/cf:cq/1697385870604/Put/vlen=4/seqid=0; \x00\x00\x00\x07/cf:cq/1697385870607/Put/vlen=4/seqid=0; \x00\x00\x00\x08/cf:cq/1697385870609/Put/vlen=4/seqid=0; \x00\x00\x00\x09/cf:cq/1697385870611/Put/vlen=4/seqid=0; \x00\x00\x00\x0A/cf:cq/1697385870613/Put/vlen=4/seqid=0; \x00\x00\x00\x0B/cf:cq/1697385870615/Put/vlen=4/seqid=0; \x00\x00\x00\x0C/cf:cq/1697385870617/Put/vlen=4/seqid=0; \x00\x00\x00\x0D/cf:cq/1697385870619/Put/vlen=4/seqid=0; \x00\x00\x00\x0E/cf:cq/1697385870621/Put/vlen=4/seqid=0; \x00\x00\x00\x0F/cf:cq/1697385870623/Put/vlen=4/seqid=0; \x00\x00\x00\x10/cf:cq/1697385870625/Put/vlen=4/seqid=0; \x00\x00\x00\x11/cf:cq/1697385870627/Put/vlen=4/seqid=0; \x00\x00\x00\x12/cf:cq/1697385870629/Put/vlen=4/seqid=0; \x00\x00\x00\x13/cf:cq/1697385870631/Put/vlen=4/seqid=0; \x00\x00\x00\x14/cf:cq/1697385870633/Put/vlen=4/seqid=0; \x00\x00\x00\x15/cf:cq/1697385870635/Put/vlen=4/seqid=0; \x00\x00\x00\x16/cf:cq/1697385870638/Put/vlen=4/seqid=0; \x00\x00\x00\x17/cf:cq/1697385870639/Put/vlen=4/seqid=0; \x00\x00\x00\x18/cf:cq/1697385870641/Put/vlen=4/seqid=0; \x00\x00\x00\x19/cf:cq/1697385870643/Put/vlen=4/seqid=0; \x00\x00\x00\x1A/cf:cq/1697385870645/Put/vlen=4/seqid=0; \x00\x00\x00\x1B/cf:cq/1697385870647/Put/vlen=4/seqid=0; \x00\x00\x00\x1C/cf:cq/1697385870649/Put/vlen=4/seqid=0; \x00\x00\x00\x1D/cf:cq/1697385870651/Put/vlen=4/seqid=0; \x00\x00\x00\x1E/cf:cq/1697385870653/Put/vlen=4/seqid=0; \x00\x00\x00\x1F/cf:cq/1697385870655/Put/vlen=4/seqid=0; \x00\x00\x00 /cf:cq/1697385870657/Put/vlen=4/seqid=0; \x00\x00\x00!/cf:cq/1697385870659/Put/vlen=4/seqid=0; \x00\x00\x00"/cf:cq/1697385870662/Put/vlen=4/seqid=0; \x00\x00\x00#/cf:cq/1697385870664/Put/vlen=4/seqid=0; \x00\x00\x00$/cf:cq/1697385870666/Put/vlen=4/seqid=0; \x00\x00\x00%/cf:cq/1697385870668/Put/vlen=4/seqid=0; \x00\x00\x00&/cf:cq/1697385870671/Put/vlen=4/seqid=0; \x00\x00\x00'/cf:cq/1697385870673/Put/vlen=4/seqid=0; \x00\x00\x00(/cf:cq/1697385870676/Put/vlen=4/seqid=0; \x00\x00\x00)/cf:cq/1697385870679/Put/vlen=4/seqid=0; \x00\x00\x00*/cf:cq/1697385870683/Put/vlen=4/seqid=0; \x00\x00\x00+/cf:cq/1697385870685/Put/vlen=4/seqid=0; \x00\x00\x00,/cf:cq/1697385870686/Put/vlen=4/seqid=0; \x00\x00\x00-/cf:cq/1697385870688/Put/vlen=4/seqid=0; \x00\x00\x00./cf:cq/1697385870690/Put/vlen=4/seqid=0; \x00\x00\x00//cf:cq/1697385870691/Put/vlen=4/seqid=0; \x00\x00\x000/cf:cq/1697385870693/Put/vlen=4/seqid=0; \x00\x00\x001/cf:cq/1697385870695/Put/vlen=4/seqid=0; \x00\x00\x002/cf:cq/1697385870697/Put/vlen=4/seqid=0; \x00\x00\x003/cf:cq/1697385870699/Put/vlen=4/seqid=0; \x00\x00\x004/cf:cq/1697385870701/Put/vlen=4/seqid=0; \x00\x00\x005/cf:cq/1697385870703/Put/vlen=4/seqid=0; \x00\x00\x006/cf:cq/1697385870704/Put/vlen=4/seqid=0; \x00\x00\x007/cf:cq/1697385870706/Put/vlen=4/seqid=0; \x00\x00\x008/cf:cq/1697385870708/Put/vlen=4/seqid=0; \x00\x00\x009/cf:cq/1697385870711/Put/vlen=4/seqid=0; \x00\x00\x00:/cf:cq/1697385870712/Put/vlen=4/seqid=0; \x00\x00\x00;/cf:cq/1697385870715/Put/vlen=4/seqid=0; \x00\x00\x00/cf:cq/1697385870720/Put/vlen=4/seqid=0; \x00\x00\x00?/cf:cq/1697385870721/Put/vlen=4/seqid=0; \x00\x00\x00@/cf:cq/1697385870723/Put/vlen=4/seqid=0; \x00\x00\x00A/cf:cq/1697385870725/Put/vlen=4/seqid=0; \x00\x00\x00B/cf:cq/1697385870727/Put/vlen=4/seqid=0; \x00\x00\x00C/cf:cq/1697385870729/Put/vlen=4/seqid=0; \x00\x00\x00D/cf:cq/1697385870731/Put/vlen=4/seqid=0; \x00\x00\x00E/cf:cq/1697385870733/Put/vlen=4/seqid=0; \x00\x00\x00F/cf:cq/1697385870734/Put/vlen=4/seqid=0; \x00\x00\x00G/cf:cq/1697385870736/Put/vlen=4/seqid=0; \x00\x00\x00H/cf:cq/1697385870738/Put/vlen=4/seqid=0; \x00\x00\x00I/cf:cq/1697385870740/Put/vlen=4/seqid=0; \x00\x00\x00J/cf:cq/1697385870742/Put/vlen=4/seqid=0; \x00\x00\x00K/cf:cq/1697385870743/Put/vlen=4/seqid=0; \x00\x00\x00L/cf:cq/1697385870745/Put/vlen=4/seqid=0; \x00\x00\x00M/cf:cq/1697385870747/Put/vlen=4/seqid=0; \x00\x00\x00N/cf:cq/1697385870749/Put/vlen=4/seqid=0; \x00\x00\x00O/cf:cq/1697385870751/Put/vlen=4/seqid=0; \x00\x00\x00P/cf:cq/1697385870753/Put/vlen=4/seqid=0; \x00\x00\x00Q/cf:cq/1697385870755/Put/vlen=4/seqid=0; \x00\x00\x00R/cf:cq/1697385870756/Put/vlen=4/seqid=0; \x00\x00\x00S/cf:cq/1697385870758/Put/vlen=4/seqid=0; \x00\x00\x00T/cf:cq/1697385870760/Put/vlen=4/seqid=0; \x00\x00\x00U/cf:cq/1697385870762/Put/vlen=4/seqid=0; \x00\x00\x00V/cf:cq/1697385870765/Put/vlen=4/seqid=0; \x00\x00\x00W/cf:cq/1697385870767/Put/vlen=4/seqid=0; \x00\x00\x00X/cf:cq/1697385870769/Put/vlen=4/seqid=0; \x00\x00\x00Y/cf:cq/1697385870771/Put/vlen=4/seqid=0; \x00\x00\x00Z/cf:cq/1697385870773/Put/vlen=4/seqid=0; \x00\x00\x00[/cf:cq/1697385870774/Put/vlen=4/seqid=0; \x00\x00\x00\x5C/cf:cq/1697385870776/Put/vlen=4/seqid=0; \x00\x00\x00]/cf:cq/1697385870778/Put/vlen=4/seqid=0; \x00\x00\x00^/cf:cq/1697385870780/Put/vlen=4/seqid=0; \x00\x00\x00_/cf:cq/1697385870783/Put/vlen=4/seqid=0; \x00\x00\x00`/cf:cq/1697385870785/Put/vlen=4/seqid=0; \x00\x00\x00a/cf:cq/1697385870786/Put/vlen=4/seqid=0; \x00\x00\x00b/cf:cq/1697385870788/Put/vlen=4/seqid=0; \x00\x00\x00c/cf:cq/1697385870789/Put/vlen=4/seqid=0; \x00\x00\x00d/cf:cq/1697385870791/Put/vlen=4/seqid=0; \x00\x00\x00e/cf:cq/1697385870795/Put/vlen=4/seqid=0; \x00\x00\x00f/cf:cq/1697385870797/Put/vlen=4/seqid=0; \x00\x00\x00g/cf:cq/1697385870798/Put/vlen=4/seqid=0; \x00\x00\x00h/cf:cq/1697385870800/Put/vlen=4/seqid=0; \x00\x00\x00i/cf:cq/1697385870801/Put/vlen=4/seqid=0; \x00\x00\x00j/cf:cq/1697385870803/Put/vlen=4/seqid=0; \x00\x00\x00k/cf:cq/1697385870805/Put/vlen=4/seqid=0; \x00\x00\x00l/cf:cq/1697385870806/Put/vlen=4/seqid=0; \x00\x00\x00m/cf:cq/1697385870808/Put/vlen=4/seqid=0; \x00\x00\x00n/cf:cq/1697385870810/Put/vlen=4/seqid=0; \x00\x00\x00o/cf:cq/1697385870812/Put/vlen=4/seqid=0; \x00\x00\x00p/cf:cq/1697385870814/Put/vlen=4/seqid=0; \x00\x00\x00q/cf:cq/1697385870816/Put/vlen=4/seqid=0; \x00\x00\x00r/cf:cq/1697385870818/Put/vlen=4/seqid=0; \x00\x00\x00s/cf:cq/1697385870820/Put/vlen=4/seqid=0; \x00\x00\x00t/cf:cq/1697385870822/Put/vlen=4/seqid=0; \x00\x00\x00u/cf:cq/1697385870824/Put/vlen=4/seqid=0; \x00\x00\x00v/cf:cq/1697385870825/Put/vlen=4/seqid=0; \x00\x00\x00w/cf:cq/1697385870827/Put/vlen=4/seqid=0; \x00\x00\x00x/cf:cq/1697385870829/Put/vlen=4/seqid=0; \x00\x00\x00y/cf:cq/1697385870830/Put/vlen=4/seqid=0; \x00\x00\x00z/cf:cq/1697385870832/Put/vlen=4/seqid=0; \x00\x00\x00{/cf:cq/1697385870834/Put/vlen=4/seqid=0; \x00\x00\x00|/cf:cq/1697385870835/Put/vlen=4/seqid=0; \x00\x00\x00}/cf:cq/1697385870837/Put/vlen=4/seqid=0; \x00\x00\x00~/cf:cq/1697385870839/Put/vlen=4/seqid=0; \x00\x00\x00\x7F/cf:cq/1697385870845/Put/vlen=4/seqid=0; \x00\x00\x00\x80/cf:cq/1697385870847/Put/vlen=4/seqid=0; \x00\x00\x00\x81/cf:cq/1697385870849/Put/vlen=4/seqid=0; \x00\x00\x00\x82/cf:cq/1697385870851/Put/vlen=4/seqid=0; \x00\x00\x00\x83/cf:cq/1697385870854/Put/vlen=4/seqid=0; \x00\x00\x00\x84/cf:cq/1697385870855/Put/vlen=4/seqid=0; \x00\x00\x00\x85/cf:cq/1697385870857/Put/vlen=4/seqid=0; \x00\x00\x00\x86/cf:cq/1697385870859/Put/vlen=4/seqid=0; \x00\x00\x00\x87/cf:cq/1697385870860/Put/vlen=4/seqid=0; \x00\x00\x00\x88/cf:cq/1697385870862/Put/vlen=4/seqid=0; \x00\x00\x00\x89/cf:cq/1697385870864/Put/vlen=4/seqid=0; \x00\x00\x00\x8A/cf:cq/1697385870866/Put/vlen=4/seqid=0; \x00\x00\x00\x8B/cf:cq/1697385870867/Put/vlen=4/seqid=0; \x00\x00\x00\x8C/cf:cq/1697385870869/Put/vlen=4/seqid=0; \x00\x00\x00\x8D/cf:cq/1697385870871/Put/vlen=4/seqid=0; \x00\x00\x00\x8E/cf:cq/1697385870873/Put/vlen=4/seqid=0; \x00\x00\x00\x8F/cf:cq/1697385870875/Put/vlen=4/seqid=0; \x00\x00\x00\x90/cf:cq/1697385870876/Put/vlen=4/seqid=0; \x00\x00\x00\x91/cf:cq/1697385870878/Put/vlen=4/seqid=0; \x00\x00\x00\x92/cf:cq/1697385870881/Put/vlen=4/seqid=0; \x00\x00\x00\x93/cf:cq/1697385870884/Put/vlen=4/seqid=0; \x00\x00\x00\x94/cf:cq/1697385870889/Put/vlen=4/seqid=0; \x00\x00\x00\x95/cf:cq/1697385870892/Put/vlen=4/seqid=0; \x00\x00\x00\x96/cf:cq/1697385870894/Put/vlen=4/seqid=0; \x00\x00\x00\x97/cf:cq/1697385870896/Put/vlen=4/seqid=0; \x00\x00\x00\x98/cf:cq/1697385870898/Put/vlen=4/seqid=0; \x00\x00\x00\x99/cf:cq/1697385870900/Put/vlen=4/seqid=0; \x00\x00\x00\x9A/cf:cq/1697385870902/Put/vlen=4/seqid=0; \x00\x00\x00\x9B/cf:cq/1697385870904/Put/vlen=4/seqid=0; \x00\x00\x00\x9C/cf:cq/1697385870906/Put/vlen=4/seqid=0; \x00\x00\x00\x9D/cf:cq/1697385870908/Put/vlen=4/seqid=0; \x00\x00\x00\x9E/cf:cq/1697385870909/Put/vlen=4/seqid=0; \x00\x00\x00\x9F/cf:cq/1697385870911/Put/vlen=4/seqid=0; \x00\x00\x00\xA0/cf:cq/1697385870915/Put/vlen=4/seqid=0; \x00\x00\x00\xA1/cf:cq/1697385870917/Put/vlen=4/seqid=0; \x00\x00\x00\xA2/cf:cq/1697385870928/Put/vlen=4/seqid=0; \x00\x00\x00\xA3/cf:cq/1697385870930/Put/vlen=4/seqid=0; \x00\x00\x00\xA4/cf:cq/1697385870932/Put/vlen=4/seqid=0; \x00\x00\x00\xA5/cf:cq/1697385870934/Put/vlen=4/seqid=0; \x00\x00\x00\xA6/cf:cq/1697385870936/Put/vlen=4/seqid=0; \x00\x00\x00\xA7/cf:cq/1697385870938/Put/vlen=4/seqid=0; \x00\x00\x00\xA8/cf:cq/1697385870940/Put/vlen=4/seqid=0; \x00\x00\x00\xA9/cf:cq/1697385870942/Put/vlen=4/seqid=0; \x00\x00\x00\xAA/cf:cq/1697385870944/Put/vlen=4/seqid=0; \x00\x00\x00\xAB/cf:cq/1697385870946/Put/vlen=4/seqid=0; \x00\x00\x00\xAC/cf:cq/1697385870948/Put/vlen=4/seqid=0; \x00\x00\x00\xAD/cf:cq/1697385870949/Put/vlen=4/seqid=0; \x00\x00\x00\xAE/cf:cq/1697385870951/Put/vlen=4/seqid=0; \x00\x00\x00\xAF/cf:cq/1697385870954/Put/vlen=4/seqid=0; \x00\x00\x00\xB0/cf:cq/1697385870956/Put/vlen=4/seqid=0; \x00\x00\x00\xB1/cf:cq/1697385870958/Put/vlen=4/seqid=0; \x00\x00\x00\xB2/cf:cq/1697385870960/Put/vlen=4/seqid=0; \x00\x00\x00\xB3/cf:cq/1697385870962/Put/vlen=4/seqid=0; \x00\x00\x00\xB4/cf:cq/1697385870964/Put/vlen=4/seqid=0; \x00\x00\x00\xB5/cf:cq/1697385870966/Put/vlen=4/seqid=0; \x00\x00\x00\xB6/cf:cq/1697385870970/Put/vlen=4/seqid=0; \x00\x00\x00\xB7/cf:cq/1697385870974/Put/vlen=4/seqid=0; \x00\x00\x00\xB8/cf:cq/1697385870976/Put/vlen=4/seqid=0; \x00\x00\x00\xB9/cf:cq/1697385870998/Put/vlen=4/seqid=0; \x00\x00\x00\xBA/cf:cq/1697385871000/Put/vlen=4/seqid=0; \x00\x00\x00\xBB/cf:cq/1697385871002/Put/vlen=4/seqid=0; \x00\x00\x00\xBC/cf:cq/1697385871005/Put/vlen=4/seqid=0; \x00\x00\x00\xBD/cf:cq/1697385871008/Put/vlen=4/seqid=0; \x00\x00\x00\xBE/cf:cq/1697385871010/Put/vlen=4/seqid=0; \x00\x00\x00\xBF/cf:cq/1697385871015/Put/vlen=4/seqid=0; \x00\x00\x00\xC0/cf:cq/1697385871020/Put/vlen=4/seqid=0; \x00\x00\x00\xC1/cf:cq/1697385871023/Put/vlen=4/seqid=0; \x00\x00\x00\xC2/cf:cq/1697385871028/Put/vlen=4/seqid=0; \x00\x00\x00\xC3/cf:cq/1697385871031/Put/vlen=4/seqid=0; \x00\x00\x00\xC4/cf:cq/1697385871034/Put/vlen=4/seqid=0; \x00\x00\x00\xC5/cf:cq/1697385871038/Put/vlen=4/seqid=0; \x00\x00\x00\xC6/cf:cq/1697385871041/Put/vlen=4/seqid=0; \x00\x00\x00\xC7/cf:cq/1697385871044/Put/vlen=4/seqid=0; \x00\x00\x00\xC8/cf:cq/1697385871048/Put/vlen=4/seqid=0; \x00\x00\x00\xC9/cf:cq/1697385871051/Put/vlen=4/seqid=0; \x00\x00\x00\xCA/cf:cq/1697385871053/Put/vlen=4/seqid=0; \x00\x00\x00\xCB/cf:cq/1697385871056/Put/vlen=4/seqid=0; \x00\x00\x00\xCC/cf:cq/1697385871058/Put/vlen=4/seqid=0; \x00\x00\x00\xCD/cf:cq/1697385871060/Put/vlen=4/seqid=0; \x00\x00\x00\xCE/cf:cq/1697385871063/Put/vlen=4/seqid=0; \x00\x00\x00\xCF/cf:cq/1697385871065/Put/vlen=4/seqid=0; \x00\x00\x00\xD0/cf:cq/1697385871068/Put/vlen=4/seqid=0; \x00\x00\x00\xD1/cf:cq/1697385871070/Put/vlen=4/seqid=0; \x00\x00\x00\xD2/cf:cq/1697385871072/Put/vlen=4/seqid=0; \x00\x00\x00\xD3/cf:cq/1697385871075/Put/vlen=4/seqid=0; \x00\x00\x00\xD4/cf:cq/1697385871077/Put/vlen=4/seqid=0; \x00\x00\x00\xD5/cf:cq/1697385871086/Put/vlen=4/seqid=0; \x00\x00\x00\xD6/cf:cq/1697385871091/Put/vlen=4/seqid=0; \x00\x00\x00\xD7/cf:cq/1697385871094/Put/vlen=4/seqid=0; \x00\x00\x00\xD8/cf:cq/1697385871097/Put/vlen=4/seqid=0; \x00\x00\x00\xD9/cf:cq/1697385871100/Put/vlen=4/seqid=0; \x00\x00\x00\xDA/cf:cq/1697385871102/Put/vlen=4/seqid=0; \x00\x00\x00\xDB/cf:cq/1697385871105/Put/vlen=4/seqid=0; \x00\x00\x00\xDC/cf:cq/1697385871107/Put/vlen=4/seqid=0; \x00\x00\x00\xDD/cf:cq/1697385871109/Put/vlen=4/seqid=0; \x00\x00\x00\xDE/cf:cq/1697385871111/Put/vlen=4/seqid=0; \x00\x00\x00\xDF/cf:cq/1697385871113/Put/vlen=4/seqid=0; \x00\x00\x00\xE0/cf:cq/1697385871115/Put/vlen=4/seqid=0; \x00\x00\x00\xE1/cf:cq/1697385871121/Put/vlen=4/seqid=0; \x00\x00\x00\xE2/cf:cq/1697385871124/Put/vlen=4/seqid=0; \x00\x00\x00\xE3/cf:cq/1697385871126/Put/vlen=4/seqid=0; \x00\x00\x00\xE4/cf:cq/1697385871129/Put/vlen=4/seqid=0; \x00\x00\x00\xE5/cf:cq/1697385871131/Put/vlen=4/seqid=0; \x00\x00\x00\xE6/cf:cq/1697385871133/Put/vlen=4/seqid=0; \x00\x00\x00\xE7/cf:cq/1697385871136/Put/vlen=4/seqid=0; \x00\x00\x00\xE8/cf:cq/1697385871138/Put/vlen=4/seqid=0; \x00\x00\x00\xE9/cf:cq/1697385871140/Put/vlen=4/seqid=0; \x00\x00\x00\xEA/cf:cq/1697385871142/Put/vlen=4/seqid=0; \x00\x00\x00\xEB/cf:cq/1697385871144/Put/vlen=4/seqid=0; \x00\x00\x00\xEC/cf:cq/1697385871146/Put/vlen=4/seqid=0; \x00\x00\x00\xED/cf:cq/1697385871148/Put/vlen=4/seqid=0; \x00\x00\x00\xEE/cf:cq/1697385871150/Put/vlen=4/seqid=0; \x00\x00\x00\xEF/cf:cq/1697385871152/Put/vlen=4/seqid=0; \x00\x00\x00\xF0/cf:cq/1697385871154/Put/vlen=4/seqid=0; \x00\x00\x00\xF1/cf:cq/1697385871155/Put/vlen=4/seqid=0; \x00\x00\x00\xF2/cf:cq/1697385871158/Put/vlen=4/seqid=0; \x00\x00\x00\xF3/cf:cq/1697385871160/Put/vlen=4/seqid=0; \x00\x00\x00\xF4/cf:cq/1697385871162/Put/vlen=4/seqid=0; \x00\x00\x00\xF5/cf:cq/1697385871164/Put/vlen=4/seqid=0; \x00\x00\x00\xF6/cf:cq/1697385871166/Put/vlen=4/seqid=0; \x00\x00\x00\xF7/cf:cq/1697385871167/Put/vlen=4/seqid=0; \x00\x00\x00\xF8/cf:cq/1697385871169/Put/vlen=4/seqid=0; \x00\x00\x00\xF9/cf:cq/1697385871170/Put/vlen=4/seqid=0; \x00\x00\x00\xFA/cf:cq/1697385871172/Put/vlen=4/seqid=0; \x00\x00\x00\xFB/cf:cq/1697385871174/Put/vlen=4/seqid=0; \x00\x00\x00\xFC/cf:cq/1697385871176/Put/vlen=4/seqid=0; \x00\x00\x00\xFD/cf:cq/1697385871178/Put/vlen=4/seqid=0; \x00\x00\x00\xFE/cf:cq/1697385871179/Put/vlen=4/seqid=0; \x00\x00\x00\xFF/cf:cq/1697385871182/Put/vlen=4/seqid=0; \x00\x00\x01\x00/cf:cq/1697385871187/Put/vlen=4/seqid=0; \x00\x00\x01\x01/cf:cq/1697385871189/Put/vlen=4/seqid=0; \x00\x00\x01\x02/cf:cq/1697385871193/Put/vlen=4/seqid=0; \x00\x00\x01\x03/cf:cq/1697385871196/Put/vlen=4/seqid=0; \x00\x00\x01\x04/cf:cq/1697385871198/Put/vlen=4/seqid=0; \x00\x00\x01\x05/cf:cq/1697385871199/Put/vlen=4/seqid=0; \x00\x00\x01\x06/cf:cq/1697385871201/Put/vlen=4/seqid=0; \x00\x00\x01\x07/cf:cq/1697385871203/Put/vlen=4/seqid=0; \x00\x00\x01\x08/cf:cq/1697385871204/Put/vlen=4/seqid=0; \x00\x00\x01\x09/cf:cq/1697385871207/Put/vlen=4/seqid=0; \x00\x00\x01\x0A/cf:cq/1697385871209/Put/vlen=4/seqid=0; \x00\x00\x01\x0B/cf:cq/1697385871211/Put/vlen=4/seqid=0; \x00\x00\x01\x0C/cf:cq/1697385871212/Put/vlen=4/seqid=0; \x00\x00\x01\x0D/cf:cq/1697385871214/Put/vlen=4/seqid=0; \x00\x00\x01\x0E/cf:cq/1697385871216/Put/vlen=4/seqid=0; \x00\x00\x01\x0F/cf:cq/1697385871218/Put/vlen=4/seqid=0; \x00\x00\x01\x10/cf:cq/1697385871221/Put/vlen=4/seqid=0; \x00\x00\x01\x11/cf:cq/1697385871223/Put/vlen=4/seqid=0; \x00\x00\x01\x12/cf:cq/1697385871229/Put/vlen=4/seqid=0; \x00\x00\x01\x13/cf:cq/1697385871231/Put/vlen=4/seqid=0; \x00\x00\x01\x14/cf:cq/1697385871233/Put/vlen=4/seqid=0; \x00\x00\x01\x15/cf:cq/1697385871235/Put/vlen=4/seqid=0; \x00\x00\x01\x16/cf:cq/1697385871237/Put/vlen=4/seqid=0; \x00\x00\x01\x17/cf:cq/1697385871239/Put/vlen=4/seqid=0; \x00\x00\x01\x18/cf:cq/1697385871241/Put/vlen=4/seqid=0; \x00\x00\x01\x19/cf:cq/1697385871243/Put/vlen=4/seqid=0; \x00\x00\x01\x1A/cf:cq/1697385871244/Put/vlen=4/seqid=0; \x00\x00\x01\x1B/cf:cq/1697385871246/Put/vlen=4/seqid=0; \x00\x00\x01\x1C/cf:cq/1697385871248/Put/vlen=4/seqid=0; \x00\x00\x01\x1D/cf:cq/1697385871250/Put/vlen=4/seqid=0; \x00\x00\x01\x1E/cf:cq/1697385871251/Put/vlen=4/seqid=0; \x00\x00\x01\x1F/cf:cq/1697385871253/Put/vlen=4/seqid=0; \x00\x00\x01 /cf:cq/1697385871257/Put/vlen=4/seqid=0; \x00\x00\x01!/cf:cq/1697385871260/Put/vlen=4/seqid=0; \x00\x00\x01"/cf:cq/1697385871261/Put/vlen=4/seqid=0; \x00\x00\x01#/cf:cq/1697385871263/Put/vlen=4/seqid=0; \x00\x00\x01$/cf:cq/1697385871264/Put/vlen=4/seqid=0; \x00\x00\x01%/cf:cq/1697385871266/Put/vlen=4/seqid=0; \x00\x00\x01&/cf:cq/1697385871268/Put/vlen=4/seqid=0; \x00\x00\x01'/cf:cq/1697385871270/Put/vlen=4/seqid=0; \x00\x00\x01(/cf:cq/1697385871271/Put/vlen=4/seqid=0; \x00\x00\x01)/cf:cq/1697385871274/Put/vlen=4/seqid=0; \x00\x00\x01*/cf:cq/1697385871278/Put/vlen=4/seqid=0; \x00\x00\x01+/cf:cq/1697385871279/Put/vlen=4/seqid=0; \x00\x00\x01,/cf:cq/1697385871282/Put/vlen=4/seqid=0; \x00\x00\x01-/cf:cq/1697385871285/Put/vlen=4/seqid=0; \x00\x00\x01./cf:cq/1697385871289/Put/vlen=4/seqid=0; \x00\x00\x01//cf:cq/1697385871291/Put/vlen=4/seqid=0; \x00\x00\x010/cf:cq/1697385871293/Put/vlen=4/seqid=0; \x00\x00\x011/cf:cq/1697385871297/Put/vlen=4/seqid=0; \x00\x00\x012/cf:cq/1697385871307/Put/vlen=4/seqid=0; \x00\x00\x013/cf:cq/1697385871310/Put/vlen=4/seqid=0; \x00\x00\x014/cf:cq/1697385871312/Put/vlen=4/seqid=0; \x00\x00\x015/cf:cq/1697385871314/Put/vlen=4/seqid=0; \x00\x00\x016/cf:cq/1697385871316/Put/vlen=4/seqid=0; \x00\x00\x017/cf:cq/1697385871318/Put/vlen=4/seqid=0; \x00\x00\x018/cf:cq/1697385871320/Put/vlen=4/seqid=0; \x00\x00\x019/cf:cq/1697385871322/Put/vlen=4/seqid=0; \x00\x00\x01:/cf:cq/1697385871325/Put/vlen=4/seqid=0; \x00\x00\x01;/cf:cq/1697385871327/Put/vlen=4/seqid=0; \x00\x00\x01/cf:cq/1697385871333/Put/vlen=4/seqid=0; \x00\x00\x01?/cf:cq/1697385871338/Put/vlen=4/seqid=0; \x00\x00\x01@/cf:cq/1697385871340/Put/vlen=4/seqid=0; \x00\x00\x01A/cf:cq/1697385871342/Put/vlen=4/seqid=0; \x00\x00\x01B/cf:cq/1697385871345/Put/vlen=4/seqid=0; \x00\x00\x01C/cf:cq/1697385871346/Put/vlen=4/seqid=0; \x00\x00\x01D/cf:cq/1697385871349/Put/vlen=4/seqid=0; \x00\x00\x01E/cf:cq/1697385871350/Put/vlen=4/seqid=0; \x00\x00\x01F/cf:cq/1697385871352/Put/vlen=4/seqid=0; \x00\x00\x01G/cf:cq/1697385871354/Put/vlen=4/seqid=0; \x00\x00\x01H/cf:cq/1697385871356/Put/vlen=4/seqid=0; \x00\x00\x01I/cf:cq/1697385871358/Put/vlen=4/seqid=0; \x00\x00\x01J/cf:cq/1697385871360/Put/vlen=4/seqid=0; \x00\x00\x01K/cf:cq/1697385871362/Put/vlen=4/seqid=0; \x00\x00\x01L/cf:cq/1697385871367/Put/vlen=4/seqid=0; \x00\x00\x01M/cf:cq/1697385871372/Put/vlen=4/seqid=0; \x00\x00\x01N/cf:cq/1697385871374/Put/vlen=4/seqid=0; \x00\x00\x01O/cf:cq/1697385871376/Put/vlen=4/seqid=0; \x00\x00\x01P/cf:cq/1697385871377/Put/vlen=4/seqid=0; \x00\x00\x01Q/cf:cq/1697385871380/Put/vlen=4/seqid=0; \x00\x00\x01R/cf:cq/1697385871381/Put/vlen=4/seqid=0; \x00\x00\x01S/cf:cq/1697385871383/Put/vlen=4/seqid=0; \x00\x00\x01T/cf:cq/1697385871385/Put/vlen=4/seqid=0; \x00\x00\x01U/cf:cq/1697385871387/Put/vlen=4/seqid=0; \x00\x00\x01V/cf:cq/1697385871389/Put/vlen=4/seqid=0; \x00\x00\x01W/cf:cq/1697385871390/Put/vlen=4/seqid=0; \x00\x00\x01X/cf:cq/1697385871394/Put/vlen=4/seqid=0; \x00\x00\x01Y/cf:cq/1697385871396/Put/vlen=4/seqid=0; \x00\x00\x01Z/cf:cq/1697385871400/Put/vlen=4/seqid=0; \x00\x00\x01[/cf:cq/1697385871402/Put/vlen=4/seqid=0; \x00\x00\x01\x5C/cf:cq/1697385871404/Put/vlen=4/seqid=0; \x00\x00\x01]/cf:cq/1697385871405/Put/vlen=4/seqid=0; \x00\x00\x01^/cf:cq/1697385871407/Put/vlen=4/seqid=0; \x00\x00\x01_/cf:cq/1697385871409/Put/vlen=4/seqid=0; \x00\x00\x01`/cf:cq/1697385871411/Put/vlen=4/seqid=0; \x00\x00\x01a/cf:cq/1697385871412/Put/vlen=4/seqid=0; \x00\x00\x01b/cf:cq/1697385871414/Put/vlen=4/seqid=0; \x00\x00\x01c/cf:cq/1697385871416/Put/vlen=4/seqid=0; \x00\x00\x01d/cf:cq/1697385871417/Put/vlen=4/seqid=0; \x00\x00\x01e/cf:cq/1697385871419/Put/vlen=4/seqid=0; \x00\x00\x01f/cf:cq/1697385871421/Put/vlen=4/seqid=0; \x00\x00\x01g/cf:cq/1697385871423/Put/vlen=4/seqid=0; \x00\x00\x01h/cf:cq/1697385871426/Put/vlen=4/seqid=0; \x00\x00\x01i/cf:cq/1697385871428/Put/vlen=4/seqid=0; \x00\x00\x01j/cf:cq/1697385871430/Put/vlen=4/seqid=0; \x00\x00\x01k/cf:cq/1697385871431/Put/vlen=4/seqid=0; \x00\x00\x01l/cf:cq/1697385871433/Put/vlen=4/seqid=0; \x00\x00\x01m/cf:cq/1697385871434/Put/vlen=4/seqid=0; \x00\x00\x01n/cf:cq/1697385871436/Put/vlen=4/seqid=0; \x00\x00\x01o/cf:cq/1697385871438/Put/vlen=4/seqid=0; \x00\x00\x01p/cf:cq/1697385871439/Put/vlen=4/seqid=0; \x00\x00\x01q/cf:cq/1697385871441/Put/vlen=4/seqid=0; \x00\x00\x01r/cf:cq/1697385871442/Put/vlen=4/seqid=0; \x00\x00\x01s/cf:cq/1697385871444/Put/vlen=4/seqid=0; \x00\x00\x01t/cf:cq/1697385871446/Put/vlen=4/seqid=0; \x00\x00\x01u/cf:cq/1697385871448/Put/vlen=4/seqid=0; \x00\x00\x01v/cf:cq/1697385871450/Put/vlen=4/seqid=0; \x00\x00\x01w/cf:cq/1697385871452/Put/vlen=4/seqid=0; \x00\x00\x01x/cf:cq/1697385871453/Put/vlen=4/seqid=0; \x00\x00\x01y/cf:cq/1697385871455/Put/vlen=4/seqid=0; \x00\x00\x01z/cf:cq/1697385871457/Put/vlen=4/seqid=0; \x00\x00\x01{/cf:cq/1697385871459/Put/vlen=4/seqid=0; \x00\x00\x01|/cf:cq/1697385871461/Put/vlen=4/seqid=0; \x00\x00\x01}/cf:cq/1697385871463/Put/vlen=4/seqid=0; \x00\x00\x01~/cf:cq/1697385871465/Put/vlen=4/seqid=0; \x00\x00\x01\x7F/cf:cq/1697385871466/Put/vlen=4/seqid=0; \x00\x00\x01\x80/cf:cq/1697385871468/Put/vlen=4/seqid=0; \x00\x00\x01\x81/cf:cq/1697385871470/Put/vlen=4/seqid=0; \x00\x00\x01\x82/cf:cq/1697385871471/Put/vlen=4/seqid=0; \x00\x00\x01\x83/cf:cq/1697385871473/Put/vlen=4/seqid=0; \x00\x00\x01\x84/cf:cq/1697385871475/Put/vlen=4/seqid=0; \x00\x00\x01\x85/cf:cq/1697385871477/Put/vlen=4/seqid=0; \x00\x00\x01\x86/cf:cq/1697385871479/Put/vlen=4/seqid=0; \x00\x00\x01\x87/cf:cq/1697385871482/Put/vlen=4/seqid=0; \x00\x00\x01\x88/cf:cq/1697385871484/Put/vlen=4/seqid=0; \x00\x00\x01\x89/cf:cq/1697385871486/Put/vlen=4/seqid=0; \x00\x00\x01\x8A/cf:cq/1697385871487/Put/vlen=4/seqid=0; \x00\x00\x01\x8B/cf:cq/1697385871489/Put/vlen=4/seqid=0; \x00\x00\x01\x8C/cf:cq/1697385871491/Put/vlen=4/seqid=0; \x00\x00\x01\x8D/cf:cq/1697385871493/Put/vlen=4/seqid=0; \x00\x00\x01\x8E/cf:cq/1697385871496/Put/vlen=4/seqid=0; \x00\x00\x01\x8F/cf:cq/1697385871498/Put/vlen=4/seqid=0; \x00\x00\x01\x90/cf:cq/1697385871505/Put/vlen=4/seqid=0; \x00\x00\x01\x91/cf:cq/1697385871508/Put/vlen=4/seqid=0; \x00\x00\x01\x92/cf:cq/1697385871510/Put/vlen=4/seqid=0; \x00\x00\x01\x93/cf:cq/1697385871517/Put/vlen=4/seqid=0; \x00\x00\x01\x94/cf:cq/1697385871519/Put/vlen=4/seqid=0; \x00\x00\x01\x95/cf:cq/1697385871521/Put/vlen=4/seqid=0; \x00\x00\x01\x96/cf:cq/1697385871523/Put/vlen=4/seqid=0; \x00\x00\x01\x97/cf:cq/1697385871525/Put/vlen=4/seqid=0; \x00\x00\x01\x98/cf:cq/1697385871530/Put/vlen=4/seqid=0; \x00\x00\x01\x99/cf:cq/1697385871532/Put/vlen=4/seqid=0; \x00\x00\x01\x9A/cf:cq/1697385871534/Put/vlen=4/seqid=0; \x00\x00\x01\x9B/cf:cq/1697385871536/Put/vlen=4/seqid=0; \x00\x00\x01\x9C/cf:cq/1697385871538/Put/vlen=4/seqid=0; \x00\x00\x01\x9D/cf:cq/1697385871540/Put/vlen=4/seqid=0; \x00\x00\x01\x9E/cf:cq/1697385871542/Put/vlen=4/seqid=0; \x00\x00\x01\x9F/cf:cq/1697385871544/Put/vlen=4/seqid=0; \x00\x00\x01\xA0/cf:cq/1697385871546/Put/vlen=4/seqid=0; \x00\x00\x01\xA1/cf:cq/1697385871549/Put/vlen=4/seqid=0; \x00\x00\x01\xA2/cf:cq/1697385871551/Put/vlen=4/seqid=0; \x00\x00\x01\xA3/cf:cq/1697385871553/Put/vlen=4/seqid=0; \x00\x00\x01\xA4/cf:cq/1697385871555/Put/vlen=4/seqid=0; \x00\x00\x01\xA5/cf:cq/1697385871557/Put/vlen=4/seqid=0; \x00\x00\x01\xA6/cf:cq/1697385871559/Put/vlen=4/seqid=0; \x00\x00\x01\xA7/cf:cq/1697385871561/Put/vlen=4/seqid=0; \x00\x00\x01\xA8/cf:cq/1697385871565/Put/vlen=4/seqid=0; \x00\x00\x01\xA9/cf:cq/1697385871566/Put/vlen=4/seqid=0; \x00\x00\x01\xAA/cf:cq/1697385871568/Put/vlen=4/seqid=0; \x00\x00\x01\xAB/cf:cq/1697385871570/Put/vlen=4/seqid=0; \x00\x00\x01\xAC/cf:cq/1697385871572/Put/vlen=4/seqid=0; \x00\x00\x01\xAD/cf:cq/1697385871574/Put/vlen=4/seqid=0; \x00\x00\x01\xAE/cf:cq/1697385871576/Put/vlen=4/seqid=0; \x00\x00\x01\xAF/cf:cq/1697385871581/Put/vlen=4/seqid=0; \x00\x00\x01\xB0/cf:cq/1697385871583/Put/vlen=4/seqid=0; \x00\x00\x01\xB1/cf:cq/1697385871585/Put/vlen=4/seqid=0; \x00\x00\x01\xB2/cf:cq/1697385871587/Put/vlen=4/seqid=0; \x00\x00\x01\xB3/cf:cq/1697385871589/Put/vlen=4/seqid=0; \x00\x00\x01\xB4/cf:cq/1697385871591/Put/vlen=4/seqid=0; \x00\x00\x01\xB5/cf:cq/1697385871593/Put/vlen=4/seqid=0; \x00\x00\x01\xB6/cf:cq/1697385871594/Put/vlen=4/seqid=0; \x00\x00\x01\xB7/cf:cq/1697385871597/Put/vlen=4/seqid=0; \x00\x00\x01\xB8/cf:cq/1697385871599/Put/vlen=4/seqid=0; \x00\x00\x01\xB9/cf:cq/1697385871600/Put/vlen=4/seqid=0; \x00\x00\x01\xBA/cf:cq/1697385871602/Put/vlen=4/seqid=0; \x00\x00\x01\xBB/cf:cq/1697385871604/Put/vlen=4/seqid=0; \x00\x00\x01\xBC/cf:cq/1697385871606/Put/vlen=4/seqid=0; \x00\x00\x01\xBD/cf:cq/1697385871618/Put/vlen=4/seqid=0; \x00\x00\x01\xBE/cf:cq/1697385871620/Put/vlen=4/seqid=0; \x00\x00\x01\xBF/cf:cq/1697385871622/Put/vlen=4/seqid=0; \x00\x00\x01\xC0/cf:cq/1697385871624/Put/vlen=4/seqid=0; \x00\x00\x01\xC1/cf:cq/1697385871626/Put/vlen=4/seqid=0; \x00\x00\x01\xC2/cf:cq/1697385871627/Put/vlen=4/seqid=0; \x00\x00\x01\xC3/cf:cq/1697385871631/Put/vlen=4/seqid=0; \x00\x00\x01\xC4/cf:cq/1697385871633/Put/vlen=4/seqid=0; \x00\x00\x01\xC5/cf:cq/1697385871634/Put/vlen=4/seqid=0; \x00\x00\x01\xC6/cf:cq/1697385871636/Put/vlen=4/seqid=0; \x00\x00\x01\xC7/cf:cq/1697385871638/Put/vlen=4/seqid=0; \x00\x00\x01\xC8/cf:cq/1697385871640/Put/vlen=4/seqid=0; \x00\x00\x01\xC9/cf:cq/1697385871642/Put/vlen=4/seqid=0; \x00\x00\x01\xCA/cf:cq/1697385871644/Put/vlen=4/seqid=0; \x00\x00\x01\xCB/cf:cq/1697385871645/Put/vlen=4/seqid=0; \x00\x00\x01\xCC/cf:cq/1697385871646/Put/vlen=4/seqid=0; \x00\x00\x01\xCD/cf:cq/1697385871648/Put/vlen=4/seqid=0; \x00\x00\x01\xCE/cf:cq/1697385871650/Put/vlen=4/seqid=0; \x00\x00\x01\xCF/cf:cq/1697385871651/Put/vlen=4/seqid=0; \x00\x00\x01\xD0/cf:cq/1697385871653/Put/vlen=4/seqid=0; \x00\x00\x01\xD1/cf:cq/1697385871654/Put/vlen=4/seqid=0; \x00\x00\x01\xD2/cf:cq/1697385871657/Put/vlen=4/seqid=0; \x00\x00\x01\xD3/cf:cq/1697385871659/Put/vlen=4/seqid=0; \x00\x00\x01\xD4/cf:cq/1697385871660/Put/vlen=4/seqid=0; \x00\x00\x01\xD5/cf:cq/1697385871662/Put/vlen=4/seqid=0; \x00\x00\x01\xD6/cf:cq/1697385871664/Put/vlen=4/seqid=0; \x00\x00\x01\xD7/cf:cq/1697385871666/Put/vlen=4/seqid=0; \x00\x00\x01\xD8/cf:cq/1697385871668/Put/vlen=4/seqid=0; \x00\x00\x01\xD9/cf:cq/1697385871669/Put/vlen=4/seqid=0; \x00\x00\x01\xDA/cf:cq/1697385871671/Put/vlen=4/seqid=0; \x00\x00\x01\xDB/cf:cq/1697385871672/Put/vlen=4/seqid=0; \x00\x00\x01\xDC/cf:cq/1697385871674/Put/vlen=4/seqid=0; \x00\x00\x01\xDD/cf:cq/1697385871676/Put/vlen=4/seqid=0; \x00\x00\x01\xDE/cf:cq/1697385871677/Put/vlen=4/seqid=0; \x00\x00\x01\xDF/cf:cq/1697385871679/Put/vlen=4/seqid=0; \x00\x00\x01\xE0/cf:cq/1697385871681/Put/vlen=4/seqid=0; \x00\x00\x01\xE1/cf:cq/1697385871682/Put/vlen=4/seqid=0; \x00\x00\x01\xE2/cf:cq/1697385871685/Put/vlen=4/seqid=0; \x00\x00\x01\xE3/cf:cq/1697385871687/Put/vlen=4/seqid=0; \x00\x00\x01\xE4/cf:cq/1697385871688/Put/vlen=4/seqid=0; \x00\x00\x01\xE5/cf:cq/1697385871689/Put/vlen=4/seqid=0; \x00\x00\x01\xE6/cf:cq/1697385871691/Put/vlen=4/seqid=0; \x00\x00\x01\xE7/cf:cq/1697385871693/Put/vlen=4/seqid=0; \x00\x00\x01\xE8/cf:cq/1697385871694/Put/vlen=4/seqid=0; \x00\x00\x01\xE9/cf:cq/1697385871696/Put/vlen=4/seqid=0; \x00\x00\x01\xEA/cf:cq/1697385871697/Put/vlen=4/seqid=0; \x00\x00\x01\xEB/cf:cq/1697385871699/Put/vlen=4/seqid=0; \x00\x00\x01\xEC/cf:cq/1697385871701/Put/vlen=4/seqid=0; \x00\x00\x01\xED/cf:cq/1697385871702/Put/vlen=4/seqid=0; \x00\x00\x01\xEE/cf:cq/1697385871704/Put/vlen=4/seqid=0; \x00\x00\x01\xEF/cf:cq/1697385871706/Put/vlen=4/seqid=0; \x00\x00\x01\xF0/cf:cq/1697385871708/Put/vlen=4/seqid=0; \x00\x00\x01\xF1/cf:cq/1697385871710/Put/vlen=4/seqid=0; \x00\x00\x01\xF2/cf:cq/1697385871711/Put/vlen=4/seqid=0; \x00\x00\x01\xF3/cf:cq/1697385871712/Put/vlen=4/seqid=0; \x00\x00\x01\xF4/cf:cq/1697385871714/Put/vlen=4/seqid=0; \x00\x00\x01\xF5/cf:cq/1697385871716/Put/vlen=4/seqid=0; \x00\x00\x01\xF6/cf:cq/1697385871718/Put/vlen=4/seqid=0; \x00\x00\x01\xF7/cf:cq/1697385871719/Put/vlen=4/seqid=0; \x00\x00\x01\xF8/cf:cq/1697385871721/Put/vlen=4/seqid=0; \x00\x00\x01\xF9/cf:cq/1697385871723/Put/vlen=4/seqid=0; \x00\x00\x01\xFA/cf:cq/1697385871725/Put/vlen=4/seqid=0; \x00\x00\x01\xFB/cf:cq/1697385871726/Put/vlen=4/seqid=0; \x00\x00\x01\xFC/cf:cq/1697385871728/Put/vlen=4/seqid=0; \x00\x00\x01\xFD/cf:cq/1697385871731/Put/vlen=4/seqid=0; \x00\x00\x01\xFE/cf:cq/1697385871734/Put/vlen=4/seqid=0; \x00\x00\x01\xFF/cf:cq/1697385871736/Put/vlen=4/seqid=0; \x00\x00\x02\x00/cf:cq/1697385871737/Put/vlen=4/seqid=0; \x00\x00\x02\x01/cf:cq/1697385871739/Put/vlen=4/seqid=0; \x00\x00\x02\x02/cf:cq/1697385871740/Put/vlen=4/seqid=0; \x00\x00\x02\x03/cf:cq/1697385871742/Put/vlen=4/seqid=0; \x00\x00\x02\x04/cf:cq/1697385871743/Put/vlen=4/seqid=0; \x00\x00\x02\x05/cf:cq/1697385871745/Put/vlen=4/seqid=0; \x00\x00\x02\x06/cf:cq/1697385871746/Put/vlen=4/seqid=0; \x00\x00\x02\x07/cf:cq/1697385871748/Put/vlen=4/seqid=0; \x00\x00\x02\x08/cf:cq/1697385871749/Put/vlen=4/seqid=0; \x00\x00\x02\x09/cf:cq/1697385871751/Put/vlen=4/seqid=0; \x00\x00\x02\x0A/cf:cq/1697385871753/Put/vlen=4/seqid=0; \x00\x00\x02\x0B/cf:cq/1697385871754/Put/vlen=4/seqid=0; \x00\x00\x02\x0C/cf:cq/1697385871756/Put/vlen=4/seqid=0; \x00\x00\x02\x0D/cf:cq/1697385871757/Put/vlen=4/seqid=0; \x00\x00\x02\x0E/cf:cq/1697385871759/Put/vlen=4/seqid=0; \x00\x00\x02\x0F/cf:cq/1697385871761/Put/vlen=4/seqid=0; \x00\x00\x02\x10/cf:cq/1697385871763/Put/vlen=4/seqid=0; \x00\x00\x02\x11/cf:cq/1697385871765/Put/vlen=4/seqid=0; \x00\x00\x02\x12/cf:cq/1697385871767/Put/vlen=4/seqid=0; \x00\x00\x02\x13/cf:cq/1697385871769/Put/vlen=4/seqid=0; \x00\x00\x02\x14/cf:cq/1697385871770/Put/vlen=4/seqid=0; \x00\x00\x02\x15/cf:cq/1697385871772/Put/vlen=4/seqid=0; \x00\x00\x02\x16/cf:cq/1697385871773/Put/vlen=4/seqid=0; \x00\x00\x02\x17/cf:cq/1697385871775/Put/vlen=4/seqid=0; \x00\x00\x02\x18/cf:cq/1697385871777/Put/vlen=4/seqid=0; \x00\x00\x02\x19/cf:cq/1697385871778/Put/vlen=4/seqid=0; \x00\x00\x02\x1A/cf:cq/1697385871780/Put/vlen=4/seqid=0; \x00\x00\x02\x1B/cf:cq/1697385871781/Put/vlen=4/seqid=0; \x00\x00\x02\x1C/cf:cq/1697385871783/Put/vlen=4/seqid=0; \x00\x00\x02\x1D/cf:cq/1697385871785/Put/vlen=4/seqid=0; \x00\x00\x02\x1E/cf:cq/1697385871786/Put/vlen=4/seqid=0; \x00\x00\x02\x1F/cf:cq/1697385871788/Put/vlen=4/seqid=0; \x00\x00\x02 /cf:cq/1697385871789/Put/vlen=4/seqid=0; \x00\x00\x02!/cf:cq/1697385871791/Put/vlen=4/seqid=0; \x00\x00\x02"/cf:cq/1697385871793/Put/vlen=4/seqid=0; \x00\x00\x02#/cf:cq/1697385871794/Put/vlen=4/seqid=0; \x00\x00\x02$/cf:cq/1697385871796/Put/vlen=4/seqid=0; \x00\x00\x02%/cf:cq/1697385871797/Put/vlen=4/seqid=0; \x00\x00\x02&/cf:cq/1697385871799/Put/vlen=4/seqid=0; \x00\x00\x02'/cf:cq/1697385871801/Put/vlen=4/seqid=0; \x00\x00\x02(/cf:cq/1697385871803/Put/vlen=4/seqid=0; \x00\x00\x02)/cf:cq/1697385871804/Put/vlen=4/seqid=0; \x00\x00\x02*/cf:cq/1697385871806/Put/vlen=4/seqid=0; \x00\x00\x02+/cf:cq/1697385871817/Put/vlen=4/seqid=0; \x00\x00\x02,/cf:cq/1697385871819/Put/vlen=4/seqid=0; \x00\x00\x02-/cf:cq/1697385871822/Put/vlen=4/seqid=0; \x00\x00\x02./cf:cq/1697385871823/Put/vlen=4/seqid=0; \x00\x00\x02//cf:cq/1697385871824/Put/vlen=4/seqid=0; \x00\x00\x020/cf:cq/1697385871826/Put/vlen=4/seqid=0; \x00\x00\x021/cf:cq/1697385871827/Put/vlen=4/seqid=0; \x00\x00\x022/cf:cq/1697385871829/Put/vlen=4/seqid=0; \x00\x00\x023/cf:cq/1697385871837/Put/vlen=4/seqid=0; \x00\x00\x024/cf:cq/1697385871839/Put/vlen=4/seqid=0; \x00\x00\x025/cf:cq/1697385871840/Put/vlen=4/seqid=0; \x00\x00\x026/cf:cq/1697385871841/Put/vlen=4/seqid=0; \x00\x00\x027/cf:cq/1697385871843/Put/vlen=4/seqid=0; \x00\x00\x028/cf:cq/1697385871844/Put/vlen=4/seqid=0; \x00\x00\x029/cf:cq/1697385871845/Put/vlen=4/seqid=0; \x00\x00\x02:/cf:cq/1697385871847/Put/vlen=4/seqid=0; \x00\x00\x02;/cf:cq/1697385871848/Put/vlen=4/seqid=0; \x00\x00\x02/cf:cq/1697385871852/Put/vlen=4/seqid=0; \x00\x00\x02?/cf:cq/1697385871854/Put/vlen=4/seqid=0; \x00\x00\x02@/cf:cq/1697385871855/Put/vlen=4/seqid=0; \x00\x00\x02A/cf:cq/1697385871856/Put/vlen=4/seqid=0; \x00\x00\x02B/cf:cq/1697385871858/Put/vlen=4/seqid=0; \x00\x00\x02C/cf:cq/1697385871859/Put/vlen=4/seqid=0; \x00\x00\x02D/cf:cq/1697385871861/Put/vlen=4/seqid=0; \x00\x00\x02E/cf:cq/1697385871862/Put/vlen=4/seqid=0; \x00\x00\x02F/cf:cq/1697385871863/Put/vlen=4/seqid=0; \x00\x00\x02G/cf:cq/1697385871864/Put/vlen=4/seqid=0; \x00\x00\x02H/cf:cq/1697385871866/Put/vlen=4/seqid=0; \x00\x00\x02I/cf:cq/1697385871867/Put/vlen=4/seqid=0; \x00\x00\x02J/cf:cq/1697385871869/Put/vlen=4/seqid=0; \x00\x00\x02K/cf:cq/1697385871870/Put/vlen=4/seqid=0; \x00\x00\x02L/cf:cq/1697385871872/Put/vlen=4/seqid=0; \x00\x00\x02M/cf:cq/1697385871873/Put/vlen=4/seqid=0; \x00\x00\x02N/cf:cq/1697385871874/Put/vlen=4/seqid=0; \x00\x00\x02O/cf:cq/1697385871877/Put/vlen=4/seqid=0; \x00\x00\x02P/cf:cq/1697385871878/Put/vlen=4/seqid=0; \x00\x00\x02Q/cf:cq/1697385871880/Put/vlen=4/seqid=0; \x00\x00\x02R/cf:cq/1697385871881/Put/vlen=4/seqid=0; \x00\x00\x02S/cf:cq/1697385871882/Put/vlen=4/seqid=0; \x00\x00\x02T/cf:cq/1697385871884/Put/vlen=4/seqid=0; \x00\x00\x02U/cf:cq/1697385871885/Put/vlen=4/seqid=0; \x00\x00\x02V/cf:cq/1697385871886/Put/vlen=4/seqid=0; \x00\x00\x02W/cf:cq/1697385871888/Put/vlen=4/seqid=0; \x00\x00\x02X/cf:cq/1697385871889/Put/vlen=4/seqid=0; \x00\x00\x02Y/cf:cq/1697385871891/Put/vlen=4/seqid=0; \x00\x00\x02Z/cf:cq/1697385871892/Put/vlen=4/seqid=0; \x00\x00\x02[/cf:cq/1697385871894/Put/vlen=4/seqid=0; \x00\x00\x02\x5C/cf:cq/1697385871895/Put/vlen=4/seqid=0; \x00\x00\x02]/cf:cq/1697385871896/Put/vlen=4/seqid=0; \x00\x00\x02^/cf:cq/1697385871898/Put/vlen=4/seqid=0; \x00\x00\x02_/cf:cq/1697385871899/Put/vlen=4/seqid=0; \x00\x00\x02`/cf:cq/1697385871900/Put/vlen=4/seqid=0; \x00\x00\x02a/cf:cq/1697385871901/Put/vlen=4/seqid=0; \x00\x00\x02b/cf:cq/1697385871903/Put/vlen=4/seqid=0; \x00\x00\x02c/cf:cq/1697385871904/Put/vlen=4/seqid=0; \x00\x00\x02d/cf:cq/1697385871905/Put/vlen=4/seqid=0; \x00\x00\x02e/cf:cq/1697385871908/Put/vlen=4/seqid=0; \x00\x00\x02f/cf:cq/1697385871910/Put/vlen=4/seqid=0; \x00\x00\x02g/cf:cq/1697385871912/Put/vlen=4/seqid=0; \x00\x00\x02h/cf:cq/1697385871913/Put/vlen=4/seqid=0; \x00\x00\x02i/cf:cq/1697385871915/Put/vlen=4/seqid=0; \x00\x00\x02j/cf:cq/1697385871916/Put/vlen=4/seqid=0; \x00\x00\x02k/cf:cq/1697385871918/Put/vlen=4/seqid=0; \x00\x00\x02l/cf:cq/1697385871920/Put/vlen=4/seqid=0; \x00\x00\x02m/cf:cq/1697385871921/Put/vlen=4/seqid=0; \x00\x00\x02n/cf:cq/1697385871923/Put/vlen=4/seqid=0; \x00\x00\x02o/cf:cq/1697385871924/Put/vlen=4/seqid=0; \x00\x00\x02p/cf:cq/1697385871925/Put/vlen=4/seqid=0; \x00\x00\x02q/cf:cq/1697385871927/Put/vlen=4/seqid=0; \x00\x00\x02r/cf:cq/1697385871928/Put/vlen=4/seqid=0; \x00\x00\x02s/cf:cq/1697385871929/Put/vlen=4/seqid=0; \x00\x00\x02t/cf:cq/1697385871930/Put/vlen=4/seqid=0; \x00\x00\x02u/cf:cq/1697385871932/Put/vlen=4/seqid=0; \x00\x00\x02v/cf:cq/1697385871933/Put/vlen=4/seqid=0; \x00\x00\x02w/cf:cq/1697385871934/Put/vlen=4/seqid=0; \x00\x00\x02x/cf:cq/1697385871935/Put/vlen=4/seqid=0; \x00\x00\x02y/cf:cq/1697385871937/Put/vlen=4/seqid=0; \x00\x00\x02z/cf:cq/1697385871938/Put/vlen=4/seqid=0; \x00\x00\x02{/cf:cq/1697385871939/Put/vlen=4/seqid=0; \x00\x00\x02|/cf:cq/1697385871940/Put/vlen=4/seqid=0; \x00\x00\x02}/cf:cq/1697385871941/Put/vlen=4/seqid=0; \x00\x00\x02~/cf:cq/1697385871942/Put/vlen=4/seqid=0; \x00\x00\x02\x7F/cf:cq/1697385871943/Put/vlen=4/seqid=0; \x00\x00\x02\x80/cf:cq/1697385871945/Put/vlen=4/seqid=0; \x00\x00\x02\x81/cf:cq/1697385871946/Put/vlen=4/seqid=0; \x00\x00\x02\x82/cf:cq/1697385871947/Put/vlen=4/seqid=0; \x00\x00\x02\x83/cf:cq/1697385871948/Put/vlen=4/seqid=0; \x00\x00\x02\x84/cf:cq/1697385871950/Put/vlen=4/seqid=0; \x00\x00\x02\x85/cf:cq/1697385871951/Put/vlen=4/seqid=0; \x00\x00\x02\x86/cf:cq/1697385871952/Put/vlen=4/seqid=0; \x00\x00\x02\x87/cf:cq/1697385871954/Put/vlen=4/seqid=0; \x00\x00\x02\x88/cf:cq/1697385871956/Put/vlen=4/seqid=0; \x00\x00\x02\x89/cf:cq/1697385871957/Put/vlen=4/seqid=0; \x00\x00\x02\x8A/cf:cq/1697385871958/Put/vlen=4/seqid=0; \x00\x00\x02\x8B/cf:cq/1697385871960/Put/vlen=4/seqid=0; \x00\x00\x02\x8C/cf:cq/1697385871962/Put/vlen=4/seqid=0; \x00\x00\x02\x8D/cf:cq/1697385871963/Put/vlen=4/seqid=0; \x00\x00\x02\x8E/cf:cq/1697385871965/Put/vlen=4/seqid=0; \x00\x00\x02\x8F/cf:cq/1697385871967/Put/vlen=4/seqid=0; \x00\x00\x02\x90/cf:cq/1697385871969/Put/vlen=4/seqid=0; \x00\x00\x02\x91/cf:cq/1697385871971/Put/vlen=4/seqid=0; \x00\x00\x02\x92/cf:cq/1697385871972/Put/vlen=4/seqid=0; \x00\x00\x02\x93/cf:cq/1697385871974/Put/vlen=4/seqid=0; \x00\x00\x02\x94/cf:cq/1697385871976/Put/vlen=4/seqid=0; \x00\x00\x02\x95/cf:cq/1697385871978/Put/vlen=4/seqid=0; \x00\x00\x02\x96/cf:cq/1697385871979/Put/vlen=4/seqid=0; \x00\x00\x02\x97/cf:cq/1697385871981/Put/vlen=4/seqid=0; \x00\x00\x02\x98/cf:cq/1697385871983/Put/vlen=4/seqid=0; \x00\x00\x02\x99/cf:cq/1697385871984/Put/vlen=4/seqid=0; \x00\x00\x02\x9A/cf:cq/1697385871986/Put/vlen=4/seqid=0; \x00\x00\x02\x9B/cf:cq/1697385871987/Put/vlen=4/seqid=0; \x00\x00\x02\x9C/cf:cq/1697385871989/Put/vlen=4/seqid=0; \x00\x00\x02\x9D/cf:cq/1697385871991/Put/vlen=4/seqid=0; \x00\x00\x02\x9E/cf:cq/1697385871993/Put/vlen=4/seqid=0; \x00\x00\x02\x9F/cf:cq/1697385871994/Put/vlen=4/seqid=0; \x00\x00\x02\xA0/cf:cq/1697385871996/Put/vlen=4/seqid=0; \x00\x00\x02\xA1/cf:cq/1697385871998/Put/vlen=4/seqid=0; \x00\x00\x02\xA2/cf:cq/1697385872000/Put/vlen=4/seqid=0; \x00\x00\x02\xA3/cf:cq/1697385872001/Put/vlen=4/seqid=0; \x00\x00\x02\xA4/cf:cq/1697385872003/Put/vlen=4/seqid=0; \x00\x00\x02\xA5/cf:cq/1697385872005/Put/vlen=4/seqid=0; \x00\x00\x02\xA6/cf:cq/1697385872007/Put/vlen=4/seqid=0; \x00\x00\x02\xA7/cf:cq/1697385872008/Put/vlen=4/seqid=0; \x00\x00\x02\xA8/cf:cq/1697385872010/Put/vlen=4/seqid=0; \x00\x00\x02\xA9/cf:cq/1697385872012/Put/vlen=4/seqid=0; \x00\x00\x02\xAA/cf:cq/1697385872013/Put/vlen=4/seqid=0; \x00\x00\x02\xAB/cf:cq/1697385872015/Put/vlen=4/seqid=0; \x00\x00\x02\xAC/cf:cq/1697385872016/Put/vlen=4/seqid=0; \x00\x00\x02\xAD/cf:cq/1697385872017/Put/vlen=4/seqid=0; \x00\x00\x02\xAE/cf:cq/1697385872018/Put/vlen=4/seqid=0; \x00\x00\x02\xAF/cf:cq/1697385872019/Put/vlen=4/seqid=0; \x00\x00\x02\xB0/cf:cq/1697385872022/Put/vlen=4/seqid=0; \x00\x00\x02\xB1/cf:cq/1697385872023/Put/vlen=4/seqid=0; \x00\x00\x02\xB2/cf:cq/1697385872025/Put/vlen=4/seqid=0; \x00\x00\x02\xB3/cf:cq/1697385872026/Put/vlen=4/seqid=0; \x00\x00\x02\xB4/cf:cq/1697385872028/Put/vlen=4/seqid=0; \x00\x00\x02\xB5/cf:cq/1697385872030/Put/vlen=4/seqid=0; \x00\x00\x02\xB6/cf:cq/1697385872032/Put/vlen=4/seqid=0; \x00\x00\x02\xB7/cf:cq/1697385872034/Put/vlen=4/seqid=0; \x00\x00\x02\xB8/cf:cq/1697385872036/Put/vlen=4/seqid=0; \x00\x00\x02\xB9/cf:cq/1697385872053/Put/vlen=4/seqid=0; \x00\x00\x02\xBA/cf:cq/1697385872055/Put/vlen=4/seqid=0; \x00\x00\x02\xBB/cf:cq/1697385872056/Put/vlen=4/seqid=0; \x00\x00\x02\xBC/cf:cq/1697385872058/Put/vlen=4/seqid=0; \x00\x00\x02\xBD/cf:cq/1697385872060/Put/vlen=4/seqid=0; \x00\x00\x02\xBE/cf:cq/1697385872061/Put/vlen=4/seqid=0; \x00\x00\x02\xBF/cf:cq/1697385872062/Put/vlen=4/seqid=0; \x00\x00\x02\xC0/cf:cq/1697385872064/Put/vlen=4/seqid=0; \x00\x00\x02\xC1/cf:cq/1697385872066/Put/vlen=4/seqid=0; \x00\x00\x02\xC2/cf:cq/1697385872067/Put/vlen=4/seqid=0; \x00\x00\x02\xC3/cf:cq/1697385872068/Put/vlen=4/seqid=0; \x00\x00\x02\xC4/cf:cq/1697385872069/Put/vlen=4/seqid=0; \x00\x00\x02\xC5/cf:cq/1697385872071/Put/vlen=4/seqid=0; \x00\x00\x02\xC6/cf:cq/1697385872074/Put/vlen=4/seqid=0; \x00\x00\x02\xC7/cf:cq/1697385872075/Put/vlen=4/seqid=0; \x00\x00\x02\xC8/cf:cq/1697385872077/Put/vlen=4/seqid=0; \x00\x00\x02\xC9/cf:cq/1697385872078/Put/vlen=4/seqid=0; \x00\x00\x02\xCA/cf:cq/1697385872080/Put/vlen=4/seqid=0; \x00\x00\x02\xCB/cf:cq/1697385872081/Put/vlen=4/seqid=0; \x00\x00\x02\xCC/cf:cq/1697385872082/Put/vlen=4/seqid=0; \x00\x00\x02\xCD/cf:cq/1697385872084/Put/vlen=4/seqid=0; \x00\x00\x02\xCE/cf:cq/1697385872085/Put/vlen=4/seqid=0; \x00\x00\x02\xCF/cf:cq/1697385872087/Put/vlen=4/seqid=0; \x00\x00\x02\xD0/cf:cq/1697385872088/Put/vlen=4/seqid=0; \x00\x00\x02\xD1/cf:cq/1697385872090/Put/vlen=4/seqid=0; \x00\x00\x02\xD2/cf:cq/1697385872091/Put/vlen=4/seqid=0; \x00\x00\x02\xD3/cf:cq/1697385872093/Put/vlen=4/seqid=0; \x00\x00\x02\xD4/cf:cq/1697385872095/Put/vlen=4/seqid=0; \x00\x00\x02\xD5/cf:cq/1697385872097/Put/vlen=4/seqid=0; \x00\x00\x02\xD6/cf:cq/1697385872098/Put/vlen=4/seqid=0; \x00\x00\x02\xD7/cf:cq/1697385872100/Put/vlen=4/seqid=0; \x00\x00\x02\xD8/cf:cq/1697385872102/Put/vlen=4/seqid=0; \x00\x00\x02\xD9/cf:cq/1697385872104/Put/vlen=4/seqid=0; \x00\x00\x02\xDA/cf:cq/1697385872106/Put/vlen=4/seqid=0; \x00\x00\x02\xDB/cf:cq/1697385872107/Put/vlen=4/seqid=0; \x00\x00\x02\xDC/cf:cq/1697385872111/Put/vlen=4/seqid=0; \x00\x00\x02\xDD/cf:cq/1697385872114/Put/vlen=4/seqid=0; \x00\x00\x02\xDE/cf:cq/1697385872115/Put/vlen=4/seqid=0; \x00\x00\x02\xDF/cf:cq/1697385872117/Put/vlen=4/seqid=0; \x00\x00\x02\xE0/cf:cq/1697385872118/Put/vlen=4/seqid=0; \x00\x00\x02\xE1/cf:cq/1697385872119/Put/vlen=4/seqid=0; \x00\x00\x02\xE2/cf:cq/1697385872121/Put/vlen=4/seqid=0; \x00\x00\x02\xE3/cf:cq/1697385872122/Put/vlen=4/seqid=0; \x00\x00\x02\xE4/cf:cq/1697385872124/Put/vlen=4/seqid=0; \x00\x00\x02\xE5/cf:cq/1697385872125/Put/vlen=4/seqid=0; \x00\x00\x02\xE6/cf:cq/1697385872127/Put/vlen=4/seqid=0; \x00\x00\x02\xE7/cf:cq/1697385872128/Put/vlen=4/seqid=0; \x00\x00\x02\xE8/cf:cq/1697385872129/Put/vlen=4/seqid=0; \x00\x00\x02\xE9/cf:cq/1697385872131/Put/vlen=4/seqid=0; \x00\x00\x02\xEA/cf:cq/1697385872132/Put/vlen=4/seqid=0; \x00\x00\x02\xEB/cf:cq/1697385872134/Put/vlen=4/seqid=0; \x00\x00\x02\xEC/cf:cq/1697385872135/Put/vlen=4/seqid=0; \x00\x00\x02\xED/cf:cq/1697385872138/Put/vlen=4/seqid=0; \x00\x00\x02\xEE/cf:cq/1697385872139/Put/vlen=4/seqid=0; \x00\x00\x02\xEF/cf:cq/1697385872140/Put/vlen=4/seqid=0; \x00\x00\x02\xF0/cf:cq/1697385872142/Put/vlen=4/seqid=0; \x00\x00\x02\xF1/cf:cq/1697385872143/Put/vlen=4/seqid=0; \x00\x00\x02\xF2/cf:cq/1697385872144/Put/vlen=4/seqid=0; \x00\x00\x02\xF3/cf:cq/1697385872146/Put/vlen=4/seqid=0; \x00\x00\x02\xF4/cf:cq/1697385872147/Put/vlen=4/seqid=0; \x00\x00\x02\xF5/cf:cq/1697385872149/Put/vlen=4/seqid=0; \x00\x00\x02\xF6/cf:cq/1697385872150/Put/vlen=4/seqid=0; \x00\x00\x02\xF7/cf:cq/1697385872152/Put/vlen=4/seqid=0; \x00\x00\x02\xF8/cf:cq/1697385872153/Put/vlen=4/seqid=0; \x00\x00\x02\xF9/cf:cq/1697385872155/Put/vlen=4/seqid=0; \x00\x00\x02\xFA/cf:cq/1697385872156/Put/vlen=4/seqid=0; \x00\x00\x02\xFB/cf:cq/1697385872157/Put/vlen=4/seqid=0; \x00\x00\x02\xFC/cf:cq/1697385872158/Put/vlen=4/seqid=0; \x00\x00\x02\xFD/cf:cq/1697385872160/Put/vlen=4/seqid=0; \x00\x00\x02\xFE/cf:cq/1697385872162/Put/vlen=4/seqid=0; \x00\x00\x02\xFF/cf:cq/1697385872164/Put/vlen=4/seqid=0; \x00\x00\x03\x00/cf:cq/1697385872166/Put/vlen=4/seqid=0; \x00\x00\x03\x01/cf:cq/1697385872167/Put/vlen=4/seqid=0; \x00\x00\x03\x02/cf:cq/1697385872169/Put/vlen=4/seqid=0; \x00\x00\x03\x03/cf:cq/1697385872170/Put/vlen=4/seqid=0; \x00\x00\x03\x04/cf:cq/1697385872171/Put/vlen=4/seqid=0; \x00\x00\x03\x05/cf:cq/1697385872173/Put/vlen=4/seqid=0; \x00\x00\x03\x06/cf:cq/1697385872175/Put/vlen=4/seqid=0; \x00\x00\x03\x07/cf:cq/1697385872176/Put/vlen=4/seqid=0; \x00\x00\x03\x08/cf:cq/1697385872178/Put/vlen=4/seqid=0; \x00\x00\x03\x09/cf:cq/1697385872179/Put/vlen=4/seqid=0; \x00\x00\x03\x0A/cf:cq/1697385872180/Put/vlen=4/seqid=0; \x00\x00\x03\x0B/cf:cq/1697385872181/Put/vlen=4/seqid=0; \x00\x00\x03\x0C/cf:cq/1697385872182/Put/vlen=4/seqid=0; \x00\x00\x03\x0D/cf:cq/1697385872184/Put/vlen=4/seqid=0; \x00\x00\x03\x0E/cf:cq/1697385872185/Put/vlen=4/seqid=0; \x00\x00\x03\x0F/cf:cq/1697385872186/Put/vlen=4/seqid=0; \x00\x00\x03\x10/cf:cq/1697385872187/Put/vlen=4/seqid=0; \x00\x00\x03\x11/cf:cq/1697385872188/Put/vlen=4/seqid=0; \x00\x00\x03\x12/cf:cq/1697385872190/Put/vlen=4/seqid=0; \x00\x00\x03\x13/cf:cq/1697385872191/Put/vlen=4/seqid=0; \x00\x00\x03\x14/cf:cq/1697385872194/Put/vlen=4/seqid=0; \x00\x00\x03\x15/cf:cq/1697385872195/Put/vlen=4/seqid=0; \x00\x00\x03\x16/cf:cq/1697385872197/Put/vlen=4/seqid=0; \x00\x00\x03\x17/cf:cq/1697385872198/Put/vlen=4/seqid=0; \x00\x00\x03\x18/cf:cq/1697385872200/Put/vlen=4/seqid=0; \x00\x00\x03\x19/cf:cq/1697385872204/Put/vlen=4/seqid=0; \x00\x00\x03\x1A/cf:cq/1697385872207/Put/vlen=4/seqid=0; \x00\x00\x03\x1B/cf:cq/1697385872208/Put/vlen=4/seqid=0; \x00\x00\x03\x1C/cf:cq/1697385872210/Put/vlen=4/seqid=0; \x00\x00\x03\x1D/cf:cq/1697385872211/Put/vlen=4/seqid=0; \x00\x00\x03\x1E/cf:cq/1697385872212/Put/vlen=4/seqid=0; \x00\x00\x03\x1F/cf:cq/1697385872214/Put/vlen=4/seqid=0; \x00\x00\x03 /cf:cq/1697385872216/Put/vlen=4/seqid=0; \x00\x00\x03!/cf:cq/1697385872217/Put/vlen=4/seqid=0; \x00\x00\x03"/cf:cq/1697385872219/Put/vlen=4/seqid=0; \x00\x00\x03#/cf:cq/1697385872224/Put/vlen=4/seqid=0; \x00\x00\x03$/cf:cq/1697385872225/Put/vlen=4/seqid=0; \x00\x00\x03%/cf:cq/1697385872227/Put/vlen=4/seqid=0; \x00\x00\x03&/cf:cq/1697385872228/Put/vlen=4/seqid=0; \x00\x00\x03'/cf:cq/1697385872229/Put/vlen=4/seqid=0; \x00\x00\x03(/cf:cq/1697385872231/Put/vlen=4/seqid=0; \x00\x00\x03)/cf:cq/1697385872232/Put/vlen=4/seqid=0; \x00\x00\x03*/cf:cq/1697385872233/Put/vlen=4/seqid=0; \x00\x00\x03+/cf:cq/1697385872234/Put/vlen=4/seqid=0; \x00\x00\x03,/cf:cq/1697385872236/Put/vlen=4/seqid=0; \x00\x00\x03-/cf:cq/1697385872237/Put/vlen=4/seqid=0; \x00\x00\x03./cf:cq/1697385872240/Put/vlen=4/seqid=0; \x00\x00\x03//cf:cq/1697385872241/Put/vlen=4/seqid=0; \x00\x00\x030/cf:cq/1697385872242/Put/vlen=4/seqid=0; \x00\x00\x031/cf:cq/1697385872244/Put/vlen=4/seqid=0; \x00\x00\x032/cf:cq/1697385872245/Put/vlen=4/seqid=0; \x00\x00\x033/cf:cq/1697385872257/Put/vlen=4/seqid=0; \x00\x00\x034/cf:cq/1697385872258/Put/vlen=4/seqid=0; \x00\x00\x035/cf:cq/1697385872260/Put/vlen=4/seqid=0; \x00\x00\x036/cf:cq/1697385872261/Put/vlen=4/seqid=0; \x00\x00\x037/cf:cq/1697385872263/Put/vlen=4/seqid=0; \x00\x00\x038/cf:cq/1697385872264/Put/vlen=4/seqid=0; \x00\x00\x039/cf:cq/1697385872267/Put/vlen=4/seqid=0; \x00\x00\x03:/cf:cq/1697385872269/Put/vlen=4/seqid=0; \x00\x00\x03;/cf:cq/1697385872270/Put/vlen=4/seqid=0; \x00\x00\x03/cf:cq/1697385872276/Put/vlen=4/seqid=0; \x00\x00\x03?/cf:cq/1697385872279/Put/vlen=4/seqid=0; \x00\x00\x03@/cf:cq/1697385872283/Put/vlen=4/seqid=0; \x00\x00\x03A/cf:cq/1697385872285/Put/vlen=4/seqid=0; \x00\x00\x03B/cf:cq/1697385872287/Put/vlen=4/seqid=0; \x00\x00\x03C/cf:cq/1697385872289/Put/vlen=4/seqid=0; \x00\x00\x03D/cf:cq/1697385872297/Put/vlen=4/seqid=0; \x00\x00\x03E/cf:cq/1697385872299/Put/vlen=4/seqid=0; \x00\x00\x03F/cf:cq/1697385872302/Put/vlen=4/seqid=0; \x00\x00\x03G/cf:cq/1697385872312/Put/vlen=4/seqid=0; \x00\x00\x03H/cf:cq/1697385872315/Put/vlen=4/seqid=0; \x00\x00\x03I/cf:cq/1697385872319/Put/vlen=4/seqid=0; \x00\x00\x03J/cf:cq/1697385872321/Put/vlen=4/seqid=0; \x00\x00\x03K/cf:cq/1697385872322/Put/vlen=4/seqid=0; \x00\x00\x03L/cf:cq/1697385872323/Put/vlen=4/seqid=0; \x00\x00\x03M/cf:cq/1697385872325/Put/vlen=4/seqid=0; \x00\x00\x03N/cf:cq/1697385872327/Put/vlen=4/seqid=0; \x00\x00\x03O/cf:cq/1697385872328/Put/vlen=4/seqid=0; \x00\x00\x03P/cf:cq/1697385872329/Put/vlen=4/seqid=0; \x00\x00\x03Q/cf:cq/1697385872330/Put/vlen=4/seqid=0; \x00\x00\x03R/cf:cq/1697385872332/Put/vlen=4/seqid=0; \x00\x00\x03S/cf:cq/1697385872333/Put/vlen=4/seqid=0; \x00\x00\x03T/cf:cq/1697385872335/Put/vlen=4/seqid=0; \x00\x00\x03U/cf:cq/1697385872336/Put/vlen=4/seqid=0; \x00\x00\x03V/cf:cq/1697385872338/Put/vlen=4/seqid=0; \x00\x00\x03W/cf:cq/1697385872340/Put/vlen=4/seqid=0; \x00\x00\x03X/cf:cq/1697385872342/Put/vlen=4/seqid=0; \x00\x00\x03Y/cf:cq/1697385872343/Put/vlen=4/seqid=0; \x00\x00\x03Z/cf:cq/1697385872344/Put/vlen=4/seqid=0; \x00\x00\x03[/cf:cq/1697385872346/Put/vlen=4/seqid=0; \x00\x00\x03\x5C/cf:cq/1697385872347/Put/vlen=4/seqid=0; \x00\x00\x03]/cf:cq/1697385872348/Put/vlen=4/seqid=0; \x00\x00\x03^/cf:cq/1697385872350/Put/vlen=4/seqid=0; \x00\x00\x03_/cf:cq/1697385872351/Put/vlen=4/seqid=0; \x00\x00\x03`/cf:cq/1697385872352/Put/vlen=4/seqid=0; \x00\x00\x03a/cf:cq/1697385872354/Put/vlen=4/seqid=0; \x00\x00\x03b/cf:cq/1697385872355/Put/vlen=4/seqid=0; \x00\x00\x03c/cf:cq/1697385872356/Put/vlen=4/seqid=0; \x00\x00\x03d/cf:cq/1697385872357/Put/vlen=4/seqid=0; \x00\x00\x03e/cf:cq/1697385872358/Put/vlen=4/seqid=0; \x00\x00\x03f/cf:cq/1697385872359/Put/vlen=4/seqid=0; \x00\x00\x03g/cf:cq/1697385872361/Put/vlen=4/seqid=0; \x00\x00\x03h/cf:cq/1697385872362/Put/vlen=4/seqid=0; \x00\x00\x03i/cf:cq/1697385872364/Put/vlen=4/seqid=0; \x00\x00\x03j/cf:cq/1697385872366/Put/vlen=4/seqid=0; \x00\x00\x03k/cf:cq/1697385872367/Put/vlen=4/seqid=0; \x00\x00\x03l/cf:cq/1697385872369/Put/vlen=4/seqid=0; \x00\x00\x03m/cf:cq/1697385872370/Put/vlen=4/seqid=0; \x00\x00\x03n/cf:cq/1697385872371/Put/vlen=4/seqid=0; \x00\x00\x03o/cf:cq/1697385872372/Put/vlen=4/seqid=0; \x00\x00\x03p/cf:cq/1697385872373/Put/vlen=4/seqid=0; \x00\x00\x03q/cf:cq/1697385872374/Put/vlen=4/seqid=0; \x00\x00\x03r/cf:cq/1697385872375/Put/vlen=4/seqid=0; \x00\x00\x03s/cf:cq/1697385872377/Put/vlen=4/seqid=0; \x00\x00\x03t/cf:cq/1697385872378/Put/vlen=4/seqid=0; \x00\x00\x03u/cf:cq/1697385872379/Put/vlen=4/seqid=0; \x00\x00\x03v/cf:cq/1697385872380/Put/vlen=4/seqid=0; \x00\x00\x03w/cf:cq/1697385872381/Put/vlen=4/seqid=0; \x00\x00\x03x/cf:cq/1697385872382/Put/vlen=4/seqid=0; \x00\x00\x03y/cf:cq/1697385872383/Put/vlen=4/seqid=0; \x00\x00\x03z/cf:cq/1697385872384/Put/vlen=4/seqid=0; \x00\x00\x03{/cf:cq/1697385872385/Put/vlen=4/seqid=0; \x00\x00\x03|/cf:cq/1697385872386/Put/vlen=4/seqid=0; \x00\x00\x03}/cf:cq/1697385872388/Put/vlen=4/seqid=0; \x00\x00\x03~/cf:cq/1697385872389/Put/vlen=4/seqid=0; \x00\x00\x03\x7F/cf:cq/1697385872390/Put/vlen=4/seqid=0; \x00\x00\x03\x80/cf:cq/1697385872391/Put/vlen=4/seqid=0; \x00\x00\x03\x81/cf:cq/1697385872392/Put/vlen=4/seqid=0; \x00\x00\x03\x82/cf:cq/1697385872393/Put/vlen=4/seqid=0; \x00\x00\x03\x83/cf:cq/1697385872394/Put/vlen=4/seqid=0; \x00\x00\x03\x84/cf:cq/1697385872395/Put/vlen=4/seqid=0; \x00\x00\x03\x85/cf:cq/1697385872396/Put/vlen=4/seqid=0; \x00\x00\x03\x86/cf:cq/1697385872397/Put/vlen=4/seqid=0; \x00\x00\x03\x87/cf:cq/1697385872399/Put/vlen=4/seqid=0; \x00\x00\x03\x88/cf:cq/1697385872400/Put/vlen=4/seqid=0; \x00\x00\x03\x89/cf:cq/1697385872401/Put/vlen=4/seqid=0; \x00\x00\x03\x8A/cf:cq/1697385872402/Put/vlen=4/seqid=0; \x00\x00\x03\x8B/cf:cq/1697385872403/Put/vlen=4/seqid=0; \x00\x00\x03\x8C/cf:cq/1697385872405/Put/vlen=4/seqid=0; \x00\x00\x03\x8D/cf:cq/1697385872406/Put/vlen=4/seqid=0; \x00\x00\x03\x8E/cf:cq/1697385872407/Put/vlen=4/seqid=0; \x00\x00\x03\x8F/cf:cq/1697385872408/Put/vlen=4/seqid=0; \x00\x00\x03\x90/cf:cq/1697385872409/Put/vlen=4/seqid=0; \x00\x00\x03\x91/cf:cq/1697385872411/Put/vlen=4/seqid=0; \x00\x00\x03\x92/cf:cq/1697385872412/Put/vlen=4/seqid=0; \x00\x00\x03\x93/cf:cq/1697385872413/Put/vlen=4/seqid=0; \x00\x00\x03\x94/cf:cq/1697385872415/Put/vlen=4/seqid=0; \x00\x00\x03\x95/cf:cq/1697385872416/Put/vlen=4/seqid=0; \x00\x00\x03\x96/cf:cq/1697385872417/Put/vlen=4/seqid=0; \x00\x00\x03\x97/cf:cq/1697385872420/Put/vlen=4/seqid=0; \x00\x00\x03\x98/cf:cq/1697385872421/Put/vlen=4/seqid=0; \x00\x00\x03\x99/cf:cq/1697385872422/Put/vlen=4/seqid=0; \x00\x00\x03\x9A/cf:cq/1697385872423/Put/vlen=4/seqid=0; \x00\x00\x03\x9B/cf:cq/1697385872425/Put/vlen=4/seqid=0; \x00\x00\x03\x9C/cf:cq/1697385872426/Put/vlen=4/seqid=0; \x00\x00\x03\x9D/cf:cq/1697385872427/Put/vlen=4/seqid=0; \x00\x00\x03\x9E/cf:cq/1697385872428/Put/vlen=4/seqid=0; \x00\x00\x03\x9F/cf:cq/1697385872429/Put/vlen=4/seqid=0; \x00\x00\x03\xA0/cf:cq/1697385872430/Put/vlen=4/seqid=0; \x00\x00\x03\xA1/cf:cq/1697385872431/Put/vlen=4/seqid=0; \x00\x00\x03\xA2/cf:cq/1697385872432/Put/vlen=4/seqid=0; \x00\x00\x03\xA3/cf:cq/1697385872433/Put/vlen=4/seqid=0; \x00\x00\x03\xA4/cf:cq/1697385872435/Put/vlen=4/seqid=0; \x00\x00\x03\xA5/cf:cq/1697385872436/Put/vlen=4/seqid=0; \x00\x00\x03\xA6/cf:cq/1697385872437/Put/vlen=4/seqid=0; \x00\x00\x03\xA7/cf:cq/1697385872438/Put/vlen=4/seqid=0; \x00\x00\x03\xA8/cf:cq/1697385872439/Put/vlen=4/seqid=0; \x00\x00\x03\xA9/cf:cq/1697385872440/Put/vlen=4/seqid=0; \x00\x00\x03\xAA/cf:cq/1697385872442/Put/vlen=4/seqid=0; \x00\x00\x03\xAB/cf:cq/1697385872443/Put/vlen=4/seqid=0; \x00\x00\x03\xAC/cf:cq/1697385872445/Put/vlen=4/seqid=0; \x00\x00\x03\xAD/cf:cq/1697385872446/Put/vlen=4/seqid=0; \x00\x00\x03\xAE/cf:cq/1697385872447/Put/vlen=4/seqid=0; \x00\x00\x03\xAF/cf:cq/1697385872448/Put/vlen=4/seqid=0; \x00\x00\x03\xB0/cf:cq/1697385872449/Put/vlen=4/seqid=0; \x00\x00\x03\xB1/cf:cq/1697385872450/Put/vlen=4/seqid=0; \x00\x00\x03\xB2/cf:cq/1697385872451/Put/vlen=4/seqid=0; \x00\x00\x03\xB3/cf:cq/1697385872453/Put/vlen=4/seqid=0; \x00\x00\x03\xB4/cf:cq/1697385872454/Put/vlen=4/seqid=0; \x00\x00\x03\xB5/cf:cq/1697385872455/Put/vlen=4/seqid=0; \x00\x00\x03\xB6/cf:cq/1697385872456/Put/vlen=4/seqid=0; \x00\x00\x03\xB7/cf:cq/1697385872458/Put/vlen=4/seqid=0; \x00\x00\x03\xB8/cf:cq/1697385872459/Put/vlen=4/seqid=0; \x00\x00\x03\xB9/cf:cq/1697385872460/Put/vlen=4/seqid=0; \x00\x00\x03\xBA/cf:cq/1697385872461/Put/vlen=4/seqid=0; \x00\x00\x03\xBB/cf:cq/1697385872462/Put/vlen=4/seqid=0; \x00\x00\x03\xBC/cf:cq/1697385872464/Put/vlen=4/seqid=0; \x00\x00\x03\xBD/cf:cq/1697385872465/Put/vlen=4/seqid=0; \x00\x00\x03\xBE/cf:cq/1697385872466/Put/vlen=4/seqid=0; \x00\x00\x03\xBF/cf:cq/1697385872467/Put/vlen=4/seqid=0; \x00\x00\x03\xC0/cf:cq/1697385872468/Put/vlen=4/seqid=0; \x00\x00\x03\xC1/cf:cq/1697385872474/Put/vlen=4/seqid=0; \x00\x00\x03\xC2/cf:cq/1697385872478/Put/vlen=4/seqid=0; \x00\x00\x03\xC3/cf:cq/1697385872482/Put/vlen=4/seqid=0; \x00\x00\x03\xC4/cf:cq/1697385872484/Put/vlen=4/seqid=0; \x00\x00\x03\xC5/cf:cq/1697385872489/Put/vlen=4/seqid=0; \x00\x00\x03\xC6/cf:cq/1697385872491/Put/vlen=4/seqid=0; \x00\x00\x03\xC7/cf:cq/1697385872494/Put/vlen=4/seqid=0; \x00\x00\x03\xC8/cf:cq/1697385872495/Put/vlen=4/seqid=0; \x00\x00\x03\xC9/cf:cq/1697385872497/Put/vlen=4/seqid=0; \x00\x00\x03\xCA/cf:cq/1697385872499/Put/vlen=4/seqid=0; \x00\x00\x03\xCB/cf:cq/1697385872501/Put/vlen=4/seqid=0; \x00\x00\x03\xCC/cf:cq/1697385872503/Put/vlen=4/seqid=0; \x00\x00\x03\xCD/cf:cq/1697385872504/Put/vlen=4/seqid=0; \x00\x00\x03\xCE/cf:cq/1697385872506/Put/vlen=4/seqid=0; \x00\x00\x03\xCF/cf:cq/1697385872508/Put/vlen=4/seqid=0; \x00\x00\x03\xD0/cf:cq/1697385872509/Put/vlen=4/seqid=0; \x00\x00\x03\xD1/cf:cq/1697385872514/Put/vlen=4/seqid=0; \x00\x00\x03\xD2/cf:cq/1697385872517/Put/vlen=4/seqid=0; \x00\x00\x03\xD3/cf:cq/1697385872518/Put/vlen=4/seqid=0; \x00\x00\x03\xD4/cf:cq/1697385872520/Put/vlen=4/seqid=0; \x00\x00\x03\xD5/cf:cq/1697385872524/Put/vlen=4/seqid=0; \x00\x00\x03\xD6/cf:cq/1697385872525/Put/vlen=4/seqid=0; \x00\x00\x03\xD7/cf:cq/1697385872526/Put/vlen=4/seqid=0; \x00\x00\x03\xD8/cf:cq/1697385872528/Put/vlen=4/seqid=0; \x00\x00\x03\xD9/cf:cq/1697385872529/Put/vlen=4/seqid=0; \x00\x00\x03\xDA/cf:cq/1697385872530/Put/vlen=4/seqid=0; \x00\x00\x03\xDB/cf:cq/1697385872531/Put/vlen=4/seqid=0; \x00\x00\x03\xDC/cf:cq/1697385872532/Put/vlen=4/seqid=0; \x00\x00\x03\xDD/cf:cq/1697385872533/Put/vlen=4/seqid=0; \x00\x00\x03\xDE/cf:cq/1697385872534/Put/vlen=4/seqid=0; \x00\x00\x03\xDF/cf:cq/1697385872536/Put/vlen=4/seqid=0; \x00\x00\x03\xE0/cf:cq/1697385872537/Put/vlen=4/seqid=0; \x00\x00\x03\xE1/cf:cq/1697385872538/Put/vlen=4/seqid=0; \x00\x00\x03\xE2/cf:cq/1697385872540/Put/vlen=4/seqid=0; \x00\x00\x03\xE3/cf:cq/1697385872542/Put/vlen=4/seqid=0; \x00\x00\x03\xE4/cf:cq/1697385872544/Put/vlen=4/seqid=0; \x00\x00\x03\xE5/cf:cq/1697385872546/Put/vlen=4/seqid=0; \x00\x00\x03\xE6/cf:cq/1697385872547/Put/vlen=4/seqid=0; \x00\x00\x03\xE7/cf:cq/1697385872548/Put/vlen=4/seqid=0; >],80117}], lastWalPath=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, lastWalPosition=36149, nbRowKeys=1000, nbHFiles=0, heapSize=80117, lastSeqIds={}, endOfFile=false,usedBufferSize=80117] 2023-10-15T16:04:35,253 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:35,310 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:35,315 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:35,353 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:35,458 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 36149, reset compression=false 2023-10-15T16:04:35,463 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=36149, beingWritten=true 2023-10-15T16:04:35,554 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:35,767 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 36149, reset compression=false 2023-10-15T16:04:35,771 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=36149, beingWritten=true 2023-10-15T16:04:35,855 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:35,896 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:35,921 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C46527%2C1697385859222]: currently replicating from: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 at position: -1 2023-10-15T16:04:35,931 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] replication.ReplicationSinkServiceImpl$ReplicationStatisticsChore(116): Sink: age in ms of last applied edit: 0, total replicated edits: 1000 2023-10-15T16:04:35,931 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=80117B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C45999%2C1697385859285]: currently replicating from: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 at position: -1 2023-10-15T16:04:36,175 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 36149, reset compression=false 2023-10-15T16:04:36,179 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=36149, beingWritten=true 2023-10-15T16:04:36,256 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:36,320 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:36,325 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:36,683 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 36149, reset compression=false 2023-10-15T16:04:36,688 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=36149, beingWritten=true 2023-10-15T16:04:36,756 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:36,896 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:37,293 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 36149, reset compression=false 2023-10-15T16:04:37,297 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=36149, beingWritten=true 2023-10-15T16:04:37,329 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:37,334 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:37,357 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:37,897 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:38,001 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 36149, reset compression=false 2023-10-15T16:04:38,005 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=36149, beingWritten=true 2023-10-15T16:04:38,057 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:38,271 WARN [HBase-Metrics2-1 {}] impl.MetricsConfig(136): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-10-15T16:04:38,338 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:38,342 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:38,810 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to pos 36149, reset compression=false 2023-10-15T16:04:38,813 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=36149, beingWritten=true 2023-10-15T16:04:38,818 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:38435 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:38435 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:38,819 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2 , the old value is region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2, error=java.net.ConnectException: Call to address=8c5d72df62de:38435 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:38435 2023-10-15T16:04:38,819 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:04:38,819 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2 from cache 2023-10-15T16:04:38,820 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:38435 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:38435 2023-10-15T16:04:38,858 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:38,897 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:38,933 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'hbase:replication', row='1-8c5d72df62de,38435,1697385859175', locateType=CURRENT is [region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,45999,1697385859285, seqNum=18] 2023-10-15T16:04:38,991 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=35}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:38,993 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=35}] wal.AbstractFSWALProvider(366): Seprated old WAL dir hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de,38435,1697385859175 not exists 2023-10-15T16:04:38,993 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=35}] regionserver.ReplicationSourceManager(843): Skip enqueuing log 8c5d72df62de%2C38435%2C1697385859175.1697385861137 because it is before the start offset 8c5d72df62de%2C38435%2C1697385859175.1697385861137:-1 2023-10-15T16:04:38,993 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=35}] regionserver.ReplicationSourceManager(888): Added source for recovered queue 1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175, number of wals to replicate: 3 2023-10-15T16:04:38,993 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=35}] regionserver.ReplicationSourceManager(891): Enqueueing log hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385865402 from recovered queue for source: 1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:38,994 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=35}] regionserver.ReplicationSourceManager(891): Enqueueing log hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.1697385867465 from recovered queue for source: 1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:38,994 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=35}] regionserver.ReplicationSourceManager(891): Enqueueing log hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385867425 from recovered queue for source: 1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:38,994 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=35}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=35 2023-10-15T16:04:38,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=35 2023-10-15T16:04:38,998 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=35, resume processing ppid=34 2023-10-15T16:04:38,999 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=35, ppid=34, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.ClaimReplicationQueueRemoteProcedure in 4.2060 sec 2023-10-15T16:04:39,006 DEBUG [PEWorker-5 {}] replication.AssignReplicationQueuesProcedure(116): Finish claiming replication queues for 8c5d72df62de,38435,1697385859175 2023-10-15T16:04:39,008 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=34, resume processing ppid=28 2023-10-15T16:04:39,008 INFO [PEWorker-1 {}] procedure.ServerCrashProcedure(285): removed crashed server 8c5d72df62de,38435,1697385859175 after splitting done 2023-10-15T16:04:39,008 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1413): Finished pid=34, ppid=28, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure in 4.3740 sec 2023-10-15T16:04:39,009 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:39,010 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=28, state=SUCCESS, hasLock=false; ServerCrashProcedure 8c5d72df62de,38435,1697385859175, splitWal=true, meta=false in 4.6830 sec 2023-10-15T16:04:39,011 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x365ad98b to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:39,012 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:39,014 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f51002a connected 2023-10-15T16:04:39,014 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@102f2c5e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:39,015 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 (queues=2) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:39,015 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C38435%2C1697385859175 2023-10-15T16:04:39,015 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:39,016 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=false 2023-10-15T16:04:39,020 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C38435%2C1697385859175.rep 2023-10-15T16:04:39,020 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C38435%2C1697385859175 2023-10-15T16:04:39,020 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:39,021 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385865402, startPosition=0, beingWritten=false 2023-10-15T16:04:39,022 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(451): EOF, closing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.1697385867465 2023-10-15T16:04:39,022 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(236): No more WAL files in queue 2023-10-15T16:04:39,022 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(118): Stopping the replication source wal reader 2023-10-15T16:04:39,022 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-10-15T16:04:39,022 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(138): Finished recovering queue for group 8c5d72df62de%2C38435%2C1697385859175 of peer 1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:39,022 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceManager(540): Done with the recovered queue 1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:39,024 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C38435%2C1697385859175.rep 2023-10-15T16:04:39,025 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceManager(553): Finished recovering queue 1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 with the following stats: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C38435%2C1697385859175.rep]: currently replicating from: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385865402 at position: -1 2023-10-15T16:04:39,026 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(451): EOF, closing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385865402 2023-10-15T16:04:39,026 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(175): Read 0 WAL entries eligible for replication 2023-10-15T16:04:39,026 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385865402, lastWalPosition=2701, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-10-15T16:04:39,026 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385867425, startPosition=0, beingWritten=false 2023-10-15T16:04:39,029 ERROR [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(452): Unexpected exception in RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 currentPath=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385867425 java.lang.NullPointerException: null at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.cleanOldLogs(ReplicationSourceManager.java:662) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:649) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceInterface.logPositionAndCleanOldLogs(ReplicationSourceInterface.java:211) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.updateLogPosition(ReplicationSourceShipper.java:266) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:158) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:119) ~[classes/:?] 2023-10-15T16:04:39,029 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(451): EOF, closing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385867425 2023-10-15T16:04:39,029 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(175): Read 0 WAL entries eligible for replication 2023-10-15T16:04:39,029 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(236): No more WAL files in queue 2023-10-15T16:04:39,029 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(118): Stopping the replication source wal reader 2023-10-15T16:04:39,030 ERROR [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.HRegionServer(2389): ***** ABORTING region server 8c5d72df62de,45999,1697385859285: Unexpected exception in RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 ***** java.lang.NullPointerException: null at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.cleanOldLogs(ReplicationSourceManager.java:662) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:649) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceInterface.logPositionAndCleanOldLogs(ReplicationSourceInterface.java:211) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.updateLogPosition(ReplicationSourceShipper.java:266) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:158) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:119) ~[classes/:?] 2023-10-15T16:04:39,030 ERROR [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.HRegionServer(2396): RegionServer abort: loaded coprocessors are: [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint] 2023-10-15T16:04:39,033 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] util.JSONBean(135): Listing beans for java.lang:type=Memory 2023-10-15T16:04:39,035 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] util.JSONBean(135): Listing beans for Hadoop:service=HBase,name=RegionServer,sub=IPC 2023-10-15T16:04:39,035 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] util.JSONBean(135): Listing beans for Hadoop:service=HBase,name=RegionServer,sub=Replication 2023-10-15T16:04:39,035 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] util.JSONBean(135): Listing beans for Hadoop:service=HBase,name=RegionServer,sub=Server 2023-10-15T16:04:39,035 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.HRegionServer(2400): Dump of metrics as JSON on abort: { "beans": [ { "name": "java.lang:type=Memory", "modelerType": "sun.management.MemoryImpl", "Verbose": false, "ObjectPendingFinalizationCount": 0, "HeapMemoryUsage": { "committed": 1048576000, "init": 1048576000, "max": 2306867200, "used": 495427584 }, "NonHeapMemoryUsage": { "committed": 190423040, "init": 7667712, "max": -1, "used": 186467952 }, "ObjectName": "java.lang:type=Memory" } ], "beans": [], "beans": [], "beans": [] } 2023-10-15T16:04:39,040 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33403 {}] master.MasterRpcServices(697): 8c5d72df62de,45999,1697385859285 reported a fatal error: ***** ABORTING region server 8c5d72df62de,45999,1697385859285: Unexpected exception in RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 ***** Cause: java.lang.NullPointerException at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.cleanOldLogs(ReplicationSourceManager.java:662) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:649) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceInterface.logPositionAndCleanOldLogs(ReplicationSourceInterface.java:211) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.updateLogPosition(ReplicationSourceShipper.java:266) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:158) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:119) 2023-10-15T16:04:39,042 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.HRegionServer(2147): ***** STOPPING region server '8c5d72df62de,45999,1697385859285' ***** 2023-10-15T16:04:39,042 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.HRegionServer(2161): STOPPED: Unexpected exception in RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:39,043 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(462): Server is shutting down, give up refreshing source for peer 1 2023-10-15T16:04:39,043 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.HeapMemoryManager(220): Stopping 2023-10-15T16:04:39,043 INFO [MemStoreFlusher.0 {}] regionserver.MemStoreFlusher$FlushHandler(374): MemStoreFlusher.0 exiting 2023-10-15T16:04:39,043 INFO [RS:2;8c5d72df62de:45999 {}] flush.RegionServerFlushTableProcedureManager(119): Stopping region server flush procedure manager abruptly. 2023-10-15T16:04:39,043 INFO [RS:2;8c5d72df62de:45999 {}] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager abruptly. 2023-10-15T16:04:39,044 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.HRegionServer(3035): Received CLOSE for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:39,044 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.HRegionServer(3035): Received CLOSE for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:39,044 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.HRegionServer(940): aborting server 8c5d72df62de,45999,1697385859285 2023-10-15T16:04:39,044 INFO [RS:2;8c5d72df62de:45999 {}] hbase.HBaseServerBase(452): Close async cluster connection 2023-10-15T16:04:39,044 INFO [RS:2;8c5d72df62de:45999 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:2;8c5d72df62de:45999. 2023-10-15T16:04:39,044 DEBUG [RS:2;8c5d72df62de:45999 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.HBaseServerBase.closeClusterConnection(HBaseServerBase.java:454) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:946) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:39,044 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1710): Closing 2d525cde81451d9cc8ed5e98268bcae5, disabling compactions & flushes 2023-10-15T16:04:39,044 DEBUG [RS:2;8c5d72df62de:45999 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:39,044 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1743): Closing region hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:39,045 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1764): Time limited wait for close lock on hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:39,045 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.CompactSplit(469): Waiting for Split Thread to finish... 2023-10-15T16:04:39,045 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1831): Acquired close lock on hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. after waiting 0 ms 2023-10-15T16:04:39,045 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.CompactSplit(469): Waiting for Large Compaction Thread to finish... 2023-10-15T16:04:39,045 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1841): Updates disabled for region hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:39,045 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.CompactSplit(469): Waiting for Small Compaction Thread to finish... 2023-10-15T16:04:39,045 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.HRegionServer(3035): Received CLOSE for 1588230740 2023-10-15T16:04:39,048 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.HRegionServer(1296): Waiting on 3 regions to close 2023-10-15T16:04:39,048 DEBUG [RS:2;8c5d72df62de:45999 {}] regionserver.HRegionServer(1300): Online Regions={1588230740=hbase:meta,,1.1588230740, 2d525cde81451d9cc8ed5e98268bcae5=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., f3bcdec0351888cb7677be57c1f453c1=SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1.} 2023-10-15T16:04:39,048 DEBUG [RS:2;8c5d72df62de:45999 {}] regionserver.HRegionServer(1326): Waiting on 1588230740, 2d525cde81451d9cc8ed5e98268bcae5, f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:39,048 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1710): Closing 1588230740, disabling compactions & flushes 2023-10-15T16:04:39,049 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1743): Closing region hbase:meta,,1.1588230740 2023-10-15T16:04:39,049 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1764): Time limited wait for close lock on hbase:meta,,1.1588230740 2023-10-15T16:04:39,050 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1831): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-10-15T16:04:39,050 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1841): Updates disabled for region hbase:meta,,1.1588230740 2023-10-15T16:04:39,052 ERROR [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1948): Memstore data size is 3599 in region hbase:meta,,1.1588230740 2023-10-15T16:04:39,052 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-15T16:04:39,052 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1961): Closed hbase:meta,,1.1588230740 2023-10-15T16:04:39,052 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1664): Region close journal for 1588230740: Waiting for close lock at 1697385879048Running coprocessor pre-close hooks at 1697385879048Disabling compacts and flushes for region at 1697385879048Disabling writes for close at 1697385879050 (+2 ms)Writing region close event to WAL at 1697385879052 (+2 ms)Running coprocessor post-close hooks at 1697385879052Closed at 1697385879052 2023-10-15T16:04:39,052 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-10-15T16:04:39,056 ERROR [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1948): Memstore data size is 521 in region hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:39,056 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-15T16:04:39,057 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1961): Closed hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:39,057 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1664): Region close journal for 2d525cde81451d9cc8ed5e98268bcae5: Waiting for close lock at 1697385879044Running coprocessor pre-close hooks at 1697385879044Disabling compacts and flushes for region at 1697385879044Disabling writes for close at 1697385879045 (+1 ms)Writing region close event to WAL at 1697385879056 (+11 ms)Running coprocessor post-close hooks at 1697385879056Closed at 1697385879057 (+1 ms) 2023-10-15T16:04:39,057 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] handler.CloseRegionHandler(117): Closed hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:39,057 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1710): Closing f3bcdec0351888cb7677be57c1f453c1, disabling compactions & flushes 2023-10-15T16:04:39,057 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1743): Closing region SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:39,057 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1764): Time limited wait for close lock on SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:39,058 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1831): Acquired close lock on SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. after waiting 0 ms 2023-10-15T16:04:39,058 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1841): Updates disabled for region SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:39,058 ERROR [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1948): Memstore data size is 32000 in region SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:39,058 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1961): Closed SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:39,058 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1664): Region close journal for f3bcdec0351888cb7677be57c1f453c1: Waiting for close lock at 1697385879057Running coprocessor pre-close hooks at 1697385879057Disabling compacts and flushes for region at 1697385879057Disabling writes for close at 1697385879058 (+1 ms)Writing region close event to WAL at 1697385879058Running coprocessor post-close hooks at 1697385879058Closed at 1697385879058 2023-10-15T16:04:39,058 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] handler.CloseRegionHandler(117): Closed SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:39,082 INFO [regionserver/8c5d72df62de:0.leaseChecker {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:39,248 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.HRegionServer(960): stopping server 8c5d72df62de,45999,1697385859285; all regions closed. 2023-10-15T16:04:39,263 DEBUG [RS:2;8c5d72df62de:45999 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:39,263 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:39,264 INFO [RS:2;8c5d72df62de:45999 {}] hbase.HBaseServerBase(435): Shutdown chores and chore service 2023-10-15T16:04:39,264 INFO [RS:2;8c5d72df62de:45999 {}] hbase.ChoreService(370): Chore service for: regionserver/8c5d72df62de:0 had [ScheduledChore name=ReplicationSinkStatistics, period=5000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=ReplicationSourceStatistics, period=5000, unit=MILLISECONDS] on shutdown 2023-10-15T16:04:39,264 INFO [RS:2;8c5d72df62de:45999 {}] hbase.HBaseServerBase(445): Shutdown executor service 2023-10-15T16:04:39,264 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(249): LogRoller exiting. 2023-10-15T16:04:39,264 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,45999,1697385859285 because: Region server is closing 2023-10-15T16:04:39,346 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:39,351 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:39,366 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f51001e, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:39,366 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f51001e, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:39,367 INFO [RS:2;8c5d72df62de:45999 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:2;8c5d72df62de:45999. 2023-10-15T16:04:39,367 DEBUG [RS:2;8c5d72df62de:45999 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:664) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:659) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:987) at org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:160) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2481) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:995) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:39,367 DEBUG [RS:2;8c5d72df62de:45999 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:39,367 DEBUG [RS:2;8c5d72df62de:45999 {}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x4871b63d to 127.0.0.1:52753 2023-10-15T16:04:39,468 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.HBaseInterClusterReplicationEndpoint(181): Waiting for peers [Source for peer 1]: Interrupted while sleeping between retries 2023-10-15T16:04:39,468 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.HBaseInterClusterReplicationEndpoint(442): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-10-15T16:04:39,468 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] regionserver.HBaseInterClusterReplicationEndpoint(181): No sinks available at peer [Source for peer 1]: Interrupted while sleeping between retries 2023-10-15T16:04:39,468 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 {}] util.Threads(127): sleep interrupted java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) ~[?:?] at org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:125) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.sleep(ReplicationSourceWALReader.java:128) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:160) ~[classes/:?] 2023-10-15T16:04:39,468 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,45999,1697385859285 terminated 2023-10-15T16:04:39,469 INFO [RS:2;8c5d72df62de:45999 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:2;8c5d72df62de:45999. 2023-10-15T16:04:39,469 DEBUG [RS:2;8c5d72df62de:45999 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSink.stopReplicationSinkServices(ReplicationSink.java:462) at org.apache.hadoop.hbase.replication.ReplicationSinkServiceImpl.stopReplicationService(ReplicationSinkServiceImpl.java:89) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2484) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:995) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:39,469 DEBUG [RS:2;8c5d72df62de:45999 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:39,469 DEBUG [RS:2;8c5d72df62de:45999 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:39,469 INFO [Registry-endpoints-refresh-end-points {}] client.RegistryEndpointsRefresher(78): Registry end points refresher loop exited. 2023-10-15T16:04:39,470 INFO [RS:2;8c5d72df62de:45999 {}] ipc.NettyRpcServer(340): Stopping server on /172.17.0.3:45999 2023-10-15T16:04:39,473 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:38213-0x101b6f97f510005, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster2/rs 2023-10-15T16:04:39,473 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster2/rs 2023-10-15T16:04:39,473 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): regionserver:45999-0x101b6f97f510009, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster2/rs/8c5d72df62de,45999,1697385859285 2023-10-15T16:04:39,473 INFO [RS:2;8c5d72df62de:45999 {}] hbase.HBaseServerBase(476): Close zookeeper 2023-10-15T16:04:39,475 INFO [RegionServerTracker-0 {}] master.RegionServerTracker(171): RegionServer ephemeral node deleted, processing expiration [8c5d72df62de,45999,1697385859285] 2023-10-15T16:04:39,476 DEBUG [RegionServerTracker-0 {}] zookeeper.RecoverableZooKeeper(221): Node /cluster2/draining/8c5d72df62de,45999,1697385859285 already deleted, retry=false 2023-10-15T16:04:39,476 INFO [RegionServerTracker-0 {}] master.ServerManager(604): Processing expiration of 8c5d72df62de,45999,1697385859285 on 8c5d72df62de,33403,1697385859093 2023-10-15T16:04:39,477 DEBUG [RegionServerTracker-0 {}] procedure2.ProcedureExecutor(1032): Stored pid=38, state=RUNNABLE:SERVER_CRASH_START, hasLock=false; ServerCrashProcedure 8c5d72df62de,45999,1697385859285, splitWal=true, meta=true 2023-10-15T16:04:39,477 INFO [RegionServerTracker-0 {}] assignment.AssignmentManager(1862): Scheduled ServerCrashProcedure pid=38 for 8c5d72df62de,45999,1697385859285 (carryingMeta=true) 8c5d72df62de,45999,1697385859285/CRASHED/regionCount=3/lock=java.util.concurrent.locks.ReentrantReadWriteLock@563a7085[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-10-15T16:04:39,478 INFO [PEWorker-2 {}] procedure.ServerCrashProcedure(163): Start pid=38, state=RUNNABLE:SERVER_CRASH_START, hasLock=true; ServerCrashProcedure 8c5d72df62de,45999,1697385859285, splitWal=true, meta=true 2023-10-15T16:04:39,479 INFO [master/8c5d72df62de:0.Chore.1 {}] master.ServerManager(1048): Writing .lastflushedseqids file at: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/.lastflushedseqids 2023-10-15T16:04:39,479 INFO [PEWorker-2 {}] procedure.ServerCrashProcedure(333): Splitting WALs pid=38, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, hasLock=true; ServerCrashProcedure 8c5d72df62de,45999,1697385859285, splitWal=true, meta=true, isMeta: true 2023-10-15T16:04:39,480 DEBUG [PEWorker-2 {}] master.MasterWalManager(317): Renamed region directory: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting 2023-10-15T16:04:39,482 INFO [PEWorker-2 {}] master.SplitWALManager(105): 8c5d72df62de,45999,1697385859285 WAL count=2, meta=true 2023-10-15T16:04:39,484 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=39, ppid=38, state=RUNNABLE:ACQUIRE_SPLIT_WAL_WORKER, hasLock=false; SplitWALProcedure 8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta}, {pid=40, ppid=38, state=RUNNABLE:ACQUIRE_SPLIT_WAL_WORKER, hasLock=false; SplitWALProcedure 8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta}] 2023-10-15T16:04:39,491 DEBUG [PEWorker-3 {}] master.SplitWALManager(158): Acquired split WAL worker=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:39,491 DEBUG [PEWorker-4 {}] master.SplitWALManager(158): Acquired split WAL worker=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:39,492 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE, hasLock=false; SplitWALRemoteProcedure 8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta, worker=8c5d72df62de,46527,1697385859222}] 2023-10-15T16:04:39,492 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=42, ppid=39, state=RUNNABLE, hasLock=false; SplitWALRemoteProcedure 8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta, worker=8c5d72df62de,46527,1697385859222}] 2023-10-15T16:04:39,575 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): regionserver:45999-0x101b6f97f510009, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:39,575 INFO [RS:2;8c5d72df62de:45999 {}] hbase.HBaseServerBase(483): Close table descriptors 2023-10-15T16:04:39,575 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): regionserver:45999-0x101b6f97f510009, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:39,575 INFO [RS:2;8c5d72df62de:45999 {}] regionserver.HRegionServer(1015): Exiting; stopping=8c5d72df62de,45999,1697385859285; zookeeper connection closed. 2023-10-15T16:04:39,575 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5aac6b33 {}] hbase.SingleProcessHBaseCluster$SingleFileSystemShutdownThread(211): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5aac6b33 2023-10-15T16:04:39,653 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46527 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.regionserver.SplitWALCallable, pid=42 2023-10-15T16:04:39,653 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46527 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.regionserver.SplitWALCallable, pid=41 2023-10-15T16:04:39,676 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] wal.WALSplitter(299): Splitting hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta, size=3.4 K (3521bytes) 2023-10-15T16:04:39,676 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] wal.WALSplitter(299): Splitting hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta, size=1.1 K (1149bytes) 2023-10-15T16:04:39,676 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta 2023-10-15T16:04:39,676 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta 2023-10-15T16:04:39,677 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta after 1ms 2023-10-15T16:04:39,677 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta after 1ms 2023-10-15T16:04:39,680 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] wal.WALSplitter(310): Open hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta took 5ms 2023-10-15T16:04:39,680 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] wal.WALSplitter(310): Open hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta took 5ms 2023-10-15T16:04:39,696 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] wal.WALSplitter(352): Last flushed sequenceid for 1588230740: last_flushed_sequence_id: 18446744073709551615 store_sequence_id { family_name: "info" sequence_id: 5 } store_sequence_id { family_name: "ns" sequence_id: 3 } store_sequence_id { family_name: "rep_barrier" sequence_id: 8 } store_sequence_id { family_name: "table" sequence_id: 6 } 2023-10-15T16:04:39,696 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] wal.WALSplitter(352): Last flushed sequenceid for 1588230740: last_flushed_sequence_id: 18446744073709551615 store_sequence_id { family_name: "info" sequence_id: 5 } store_sequence_id { family_name: "ns" sequence_id: 3 } store_sequence_id { family_name: "rep_barrier" sequence_id: 8 } store_sequence_id { family_name: "table" sequence_id: 6 } 2023-10-15T16:04:39,696 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] wal.WALSplitter(406): Finishing writing output for hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta so closing down 2023-10-15T16:04:39,697 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] wal.OutputSink(108): Waiting for split writer threads to finish 2023-10-15T16:04:39,697 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] wal.WALSplitter(406): Finishing writing output for hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta so closing down 2023-10-15T16:04:39,697 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] wal.OutputSink(108): Waiting for split writer threads to finish 2023-10-15T16:04:39,703 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0-Writer-0 {}] monitor.StreamSlowMonitor(122): New stream slow monitor 0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta.temp 2023-10-15T16:04:39,703 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1-Writer-0 {}] monitor.StreamSlowMonitor(122): New stream slow monitor 0000000000000000016-8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta.temp 2023-10-15T16:04:39,704 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0-Writer-0 {}] wal.AbstractRecoveredEditsOutputSink(71): Creating recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta.temp 2023-10-15T16:04:39,705 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] wal.OutputSink(128): 3 split writer threads finished 2023-10-15T16:04:39,706 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1-Writer-0 {}] wal.AbstractRecoveredEditsOutputSink(71): Creating recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000016-8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta.temp 2023-10-15T16:04:39,706 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] wal.OutputSink(128): 3 split writer threads finished 2023-10-15T16:04:39,723 INFO [split-log-closeStream-pool-0 {}] wal.AbstractRecoveredEditsOutputSink(90): Closed recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta.temp (wrote 12 edits, skipped 0 edits in 0 ms) 2023-10-15T16:04:39,723 INFO [split-log-closeStream-pool-0 {}] wal.AbstractRecoveredEditsOutputSink(90): Closed recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000016-8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta.temp (wrote 2 edits, skipped 0 edits in 0 ms) 2023-10-15T16:04:39,725 INFO [split-log-closeStream-pool-0 {}] wal.AbstractRecoveredEditsOutputSink(123): Rename recovered edits hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000016-8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta.temp to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000017 2023-10-15T16:04:39,725 INFO [split-log-closeStream-pool-0 {}] wal.AbstractRecoveredEditsOutputSink(123): Rename recovered edits hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta.temp to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000015 2023-10-15T16:04:39,725 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] wal.WALSplitter(425): Processed 2 edits across 1 Regions in 44 ms; skipped=0; WAL=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta, size=1.1 K, length=1149, corrupted=false, cancelled=false 2023-10-15T16:04:39,725 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] wal.WALSplitter(425): Processed 13 edits across 1 Regions in 43 ms; skipped=1; WAL=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta, size=3.4 K, length=3521, corrupted=false, cancelled=false 2023-10-15T16:04:39,725 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] wal.WALSplitter(428): Completed split of hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta, journal: Splitting hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta, size=1.1 K (1149bytes) at 1697385879676Finishing writing output for hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta so closing down at 1697385879697 (+21 ms)Creating recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000016-8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta.temp at 1697385879706 (+9 ms)3 split writer threads finished at 1697385879706Closed recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000016-8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta.temp (wrote 2 edits, skipped 0 edits in 0 ms) at 1697385879723 (+17 ms)Rename recovered edits hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000016-8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta.temp to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000017 at 1697385879725 (+2 ms)Processed 2 edits across 1 Regions in 44 ms; skipped=0; WAL=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta, size=1.1 K, length=1149, corrupted=false, cancelled=false at 1697385879725 2023-10-15T16:04:39,725 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] wal.WALSplitter(428): Completed split of hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta, journal: Splitting hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta, size=3.4 K (3521bytes) at 1697385879676Finishing writing output for hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta so closing down at 1697385879697 (+21 ms)Creating recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta.temp at 1697385879704 (+7 ms)3 split writer threads finished at 1697385879705 (+1 ms)Closed recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta.temp (wrote 12 edits, skipped 0 edits in 0 ms) at 1697385879723 (+18 ms)Rename recovered edits hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta.temp to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000015 at 1697385879725 (+2 ms)Processed 13 edits across 1 Regions in 43 ms; skipped=1; WAL=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta, size=3.4 K, length=3521, corrupted=false, cancelled=false at 1697385879725 2023-10-15T16:04:39,725 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] regionserver.SplitLogWorker(218): Done splitting WAL hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta 2023-10-15T16:04:39,725 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] regionserver.SplitLogWorker(218): Done splitting WAL hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta 2023-10-15T16:04:39,725 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=42}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=42 2023-10-15T16:04:39,725 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=41}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=41 2023-10-15T16:04:39,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=42 2023-10-15T16:04:39,727 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] wal.WALSplitUtil(143): Moved hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs 2023-10-15T16:04:39,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=41 2023-10-15T16:04:39,729 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] wal.WALSplitUtil(143): Moved hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs 2023-10-15T16:04:39,732 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=42, resume processing ppid=39 2023-10-15T16:04:39,733 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1413): Finished pid=42, ppid=39, state=SUCCESS, hasLock=false; SplitWALRemoteProcedure 8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta, worker=8c5d72df62de,46527,1697385859222 in 237 msec 2023-10-15T16:04:39,734 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=41, resume processing ppid=40 2023-10-15T16:04:39,734 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=41, ppid=40, state=SUCCESS, hasLock=false; SplitWALRemoteProcedure 8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta, worker=8c5d72df62de,46527,1697385859222 in 240 msec 2023-10-15T16:04:39,734 DEBUG [PEWorker-3 {}] master.SplitWALManager(172): Release split WAL worker=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:39,735 DEBUG [PEWorker-1 {}] master.SplitWALManager(172): Release split WAL worker=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:39,736 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=39, ppid=38, state=SUCCESS, hasLock=false; SplitWALProcedure 8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta, worker=8c5d72df62de,46527,1697385859222 in 251 msec 2023-10-15T16:04:39,741 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=40, resume processing ppid=38 2023-10-15T16:04:39,741 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=40, ppid=38, state=SUCCESS, hasLock=false; SplitWALProcedure 8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta, worker=8c5d72df62de,46527,1697385859222 in 252 msec 2023-10-15T16:04:39,742 INFO [PEWorker-5 {}] master.SplitLogManager(171): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting dir is empty, no logs to split. 2023-10-15T16:04:39,742 INFO [PEWorker-5 {}] master.SplitWALManager(105): 8c5d72df62de,45999,1697385859285 WAL count=0, meta=true 2023-10-15T16:04:39,742 DEBUG [PEWorker-5 {}] procedure.ServerCrashProcedure(323): Check if 8c5d72df62de,45999,1697385859285 WAL splitting is done? wals=0, meta=true 2023-10-15T16:04:39,743 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=43, ppid=38, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-10-15T16:04:39,744 INFO [PEWorker-2 {}] procedure.MasterProcedureScheduler(792): Took xlock for pid=43, ppid=38, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-10-15T16:04:39,745 INFO [PEWorker-2 {}] assignment.TransitRegionStateProcedure(262): Starting pid=43, ppid=38, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-10-15T16:04:39,895 DEBUG [8c5d72df62de:33403 {}] assignment.AssignmentManager(2316): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-10-15T16:04:39,895 INFO [PEWorker-4 {}] assignment.RegionStateStore(223): pid=43 updating hbase:meta row=1588230740, regionState=OPENING, regionLocation=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:39,896 INFO [PEWorker-4 {}] zookeeper.MetaTableLocator(171): Setting hbase:meta replicaId=0 location in ZooKeeper as 8c5d72df62de,46527,1697385859222, state=OPENING 2023-10-15T16:04:39,897 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:39,899 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): regionserver:46527-0x101b6f97f510008, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:39,899 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:39,899 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:38213-0x101b6f97f510005, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:39,899 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE, hasLock=false; OpenRegionProcedure 1588230740, server=8c5d72df62de,46527,1697385859222}] 2023-10-15T16:04:39,899 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:39,899 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:39,899 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:40,055 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] handler.AssignRegionHandler(127): Open hbase:meta,,1.1588230740 2023-10-15T16:04:40,055 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] wal.WALFactory(196): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-10-15T16:04:40,056 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] monitor.StreamSlowMonitor(122): New stream slow monitor .meta 2023-10-15T16:04:40,057 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] wal.AbstractFSWAL(602): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=8c5d72df62de%2C46527%2C1697385859222.meta, suffix=.meta, logDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222, archiveDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs, maxLogs=10 2023-10-15T16:04:40,077 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.meta.1697385880058.meta, exclude list is [], retry=0 2023-10-15T16:04:40,079 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:40,079 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:40,079 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:40,081 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] wal.AbstractFSWAL(980): New WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.meta.1697385880058.meta 2023-10-15T16:04:40,082 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK], DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK], DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK]] 2023-10-15T16:04:40,082 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(7727): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-10-15T16:04:40,082 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-10-15T16:04:40,082 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(8255): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-10-15T16:04:40,082 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.RegionCoprocessorHost(391): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-10-15T16:04:40,083 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-10-15T16:04:40,083 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(891): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=0 ; minColumnNum=100 ; preparePutThreadLimit=0 ; hotProtect now disable 2023-10-15T16:04:40,083 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(7769): checking encryption for 1588230740 2023-10-15T16:04:40,083 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(7772): checking classloading for 1588230740 2023-10-15T16:04:40,084 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-10-15T16:04:40,085 INFO [StoreOpener-1588230740-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-10-15T16:04:40,085 DEBUG [StoreOpener-1588230740-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:40,086 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(324): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=ROW_INDEX_V1, compression=NONE 2023-10-15T16:04:40,086 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family ns of region 1588230740 2023-10-15T16:04:40,087 INFO [StoreOpener-1588230740-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName ns 2023-10-15T16:04:40,087 DEBUG [StoreOpener-1588230740-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:40,087 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(324): Store=1588230740/ns, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=ROW_INDEX_V1, compression=NONE 2023-10-15T16:04:40,088 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-10-15T16:04:40,088 INFO [StoreOpener-1588230740-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-10-15T16:04:40,088 DEBUG [StoreOpener-1588230740-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:40,089 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(324): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=ROW_INDEX_V1, compression=NONE 2023-10-15T16:04:40,089 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-10-15T16:04:40,090 INFO [StoreOpener-1588230740-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-10-15T16:04:40,090 DEBUG [StoreOpener-1588230740-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:40,090 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(324): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=ROW_INDEX_V1, compression=NONE 2023-10-15T16:04:40,090 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(1028): replaying wal for 1588230740 2023-10-15T16:04:40,091 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740 2023-10-15T16:04:40,093 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(5511): Found 2 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740 2023-10-15T16:04:40,094 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(5578): Replaying edits from hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000015 2023-10-15T16:04:40,099 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(5769): Applied 27, skipped 0, firstSequenceIdInLog=4, maxSequenceIdInLog=15, path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000015 2023-10-15T16:04:40,100 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(5578): Replaying edits from hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000017 2023-10-15T16:04:40,103 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(5769): Applied 8, skipped 0, firstSequenceIdInLog=16, maxSequenceIdInLog=17, path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000017 2023-10-15T16:04:40,103 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(2885): Flushing 1588230740 4/4 column families, dataSize=3.51 KB heapSize=7.10 KB 2023-10-15T16:04:40,118 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/info/846f92233df94355b7178307130e3f39 is 147, key is hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5./info:regioninfo/1697385874621/Put/seqid=0 2023-10-15T16:04:40,131 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=3.16 KB at sequenceid=17 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/info/846f92233df94355b7178307130e3f39 2023-10-15T16:04:40,153 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/ns/4551d862e332470c8a8a259837c35f56 is 43, key is default/ns:d/1697385861579/Put/seqid=0 2023-10-15T16:04:40,160 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=74 B at sequenceid=17 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/ns/4551d862e332470c8a8a259837c35f56 2023-10-15T16:04:40,180 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/rep_barrier/1243340783024e4996a370ac9696d649 is 115, key is SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1./rep_barrier:seqnumDuringOpen/1697385863468/Put/seqid=0 2023-10-15T16:04:40,187 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=111 B at sequenceid=17 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/rep_barrier/1243340783024e4996a370ac9696d649 2023-10-15T16:04:40,207 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/table/e4e23e8fab1543999e04dda4f557f657 is 53, key is hbase:replication/table:state/1697385865501/Put/seqid=0 2023-10-15T16:04:40,215 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=176 B at sequenceid=17 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/table/e4e23e8fab1543999e04dda4f557f657 2023-10-15T16:04:40,222 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/info/846f92233df94355b7178307130e3f39 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/info/846f92233df94355b7178307130e3f39 2023-10-15T16:04:40,230 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/info/846f92233df94355b7178307130e3f39, entries=28, sequenceid=17, filesize=8.5 K 2023-10-15T16:04:40,231 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/ns/4551d862e332470c8a8a259837c35f56 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/ns/4551d862e332470c8a8a259837c35f56 2023-10-15T16:04:40,238 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/ns/4551d862e332470c8a8a259837c35f56, entries=2, sequenceid=17, filesize=5.0 K 2023-10-15T16:04:40,240 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/rep_barrier/1243340783024e4996a370ac9696d649 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/rep_barrier/1243340783024e4996a370ac9696d649 2023-10-15T16:04:40,247 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/rep_barrier/1243340783024e4996a370ac9696d649, entries=1, sequenceid=17, filesize=5.4 K 2023-10-15T16:04:40,248 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/table/e4e23e8fab1543999e04dda4f557f657 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/table/e4e23e8fab1543999e04dda4f557f657 2023-10-15T16:04:40,255 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/table/e4e23e8fab1543999e04dda4f557f657, entries=4, sequenceid=17, filesize=5.2 K 2023-10-15T16:04:40,256 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(3123): Finished flush of dataSize ~3.51 KB/3599, heapSize ~7.04 KB/7208, currentSize=0 B/0 for 1588230740 in 153ms, sequenceid=17, compaction requested=false; wal=null 2023-10-15T16:04:40,256 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.MetricsTableSourceImpl(133): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-10-15T16:04:40,258 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(5388): Deleted recovered.edits file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000015 2023-10-15T16:04:40,258 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(5388): Deleted recovered.edits file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/0000000000000000017 2023-10-15T16:04:40,260 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(1038): stopping wal replay for 1588230740 2023-10-15T16:04:40,260 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(1050): Cleaning up temporary data for 1588230740 2023-10-15T16:04:40,261 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (32.0 M)) instead. 2023-10-15T16:04:40,263 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(1083): writing seq id for 1588230740 2023-10-15T16:04:40,265 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] wal.WALSplitUtil(410): Wrote file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-10-15T16:04:40,266 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(1104): Opened 1588230740; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71090146, jitterRate=0.059325724840164185}}}, FlushLargeStoresPolicy{flushSizeLowerBound=33554432} 2023-10-15T16:04:40,266 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(1112): Running coprocessor post-open hooks for 1588230740 2023-10-15T16:04:40,267 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegion(996): Region open journal for 1588230740: Running coprocessor pre-open hook at 1697385880083Writing region info on filesystem at 1697385880083Initializing all the Stores at 1697385880084 (+1 ms)Instantiating store for column family {NAME => 'info', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'ROW_INDEX_V1', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROWCOL', IN_MEMORY => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '8192 B (8KB)'} at 1697385880084Instantiating store for column family {NAME => 'ns', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'ROW_INDEX_V1', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROWCOL', IN_MEMORY => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '8192 B (8KB)'} at 1697385880084Instantiating store for column family {NAME => 'rep_barrier', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'ROW_INDEX_V1', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROWCOL', IN_MEMORY => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385880084Instantiating store for column family {NAME => 'table', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'ROW_INDEX_V1', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROWCOL', IN_MEMORY => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '8192 B (8KB)'} at 1697385880084Obtaining lock to block concurrent updates at 1697385880103 (+19 ms)Preparing flush snapshotting stores in 1588230740 at 1697385880103Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=3599, getHeapSize=7208, getOffHeapSize=0, getCellsCount=35 at 1697385880103Flushing stores of hbase:meta,,1.1588230740 at 1697385880103Flushing 1588230740/info: creating writer at 1697385880104 (+1 ms)Flushing 1588230740/info: appending metadata at 1697385880117 (+13 ms)Flushing 1588230740/info: closing flushed file at 1697385880117Flushing 1588230740/ns: creating writer at 1697385880136 (+19 ms)Flushing 1588230740/ns: appending metadata at 1697385880152 (+16 ms)Flushing 1588230740/ns: closing flushed file at 1697385880152Flushing 1588230740/rep_barrier: creating writer at 1697385880166 (+14 ms)Flushing 1588230740/rep_barrier: appending metadata at 1697385880179 (+13 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1697385880179Flushing 1588230740/table: creating writer at 1697385880193 (+14 ms)Flushing 1588230740/table: appending metadata at 1697385880206 (+13 ms)Flushing 1588230740/table: closing flushed file at 1697385880206Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@601c7d1f: reopening flushed file at 1697385880221 (+15 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@5391c420: reopening flushed file at 1697385880230 (+9 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@2c07d8d1: reopening flushed file at 1697385880238 (+8 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@788749ba: reopening flushed file at 1697385880248 (+10 ms)Finished flush of dataSize ~3.51 KB/3599, heapSize ~7.04 KB/7208, currentSize=0 B/0 for 1588230740 in 153ms, sequenceid=17, compaction requested=false; wal=null at 1697385880256 (+8 ms)Cleaning up temporary data from old regions at 1697385880260 (+4 ms)Running coprocessor post-open hooks at 1697385880266 (+6 ms)Region opened successfully at 1697385880267 (+1 ms) 2023-10-15T16:04:40,268 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegionServer(2186): Post open deploy tasks for hbase:meta,,1.1588230740, pid=44, masterSystemTime=1697385880051 2023-10-15T16:04:40,270 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] regionserver.HRegionServer(2216): Finished post open deploy task for hbase:meta,,1.1588230740 2023-10-15T16:04:40,270 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=44}] handler.AssignRegionHandler(147): Opened hbase:meta,,1.1588230740 2023-10-15T16:04:40,270 INFO [PEWorker-1 {}] assignment.RegionStateStore(223): pid=43 updating hbase:meta row=1588230740, regionState=OPEN, openSeqNum=18, regionLocation=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:40,271 INFO [PEWorker-1 {}] zookeeper.MetaTableLocator(171): Setting hbase:meta replicaId=0 location in ZooKeeper as 8c5d72df62de,46527,1697385859222, state=OPEN 2023-10-15T16:04:40,272 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:40,272 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): regionserver:46527-0x101b6f97f510008, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:40,272 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:38213-0x101b6f97f510005, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:40,273 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:40,273 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:40,273 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:40,275 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=44, resume processing ppid=43 2023-10-15T16:04:40,275 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=44, ppid=43, state=SUCCESS, hasLock=false; OpenRegionProcedure 1588230740, server=8c5d72df62de,46527,1697385859222 in 373 msec 2023-10-15T16:04:40,277 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=43, resume processing ppid=38 2023-10-15T16:04:40,277 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1413): Finished pid=43, ppid=38, state=SUCCESS, hasLock=false; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 532 msec 2023-10-15T16:04:40,277 INFO [PEWorker-2 {}] procedure.ServerCrashProcedure(201): 8c5d72df62de,45999,1697385859285 had 3 regions 2023-10-15T16:04:40,278 INFO [PEWorker-2 {}] procedure.ServerCrashProcedure(333): Splitting WALs pid=38, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, hasLock=true; ServerCrashProcedure 8c5d72df62de,45999,1697385859285, splitWal=true, meta=true, isMeta: false 2023-10-15T16:04:40,279 INFO [PEWorker-2 {}] master.SplitWALManager(105): 8c5d72df62de,45999,1697385859285 WAL count=2, meta=false 2023-10-15T16:04:40,280 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=45, ppid=38, state=RUNNABLE:ACQUIRE_SPLIT_WAL_WORKER, hasLock=false; SplitWALProcedure 8c5d72df62de%2C45999%2C1697385859285.1697385867224}, {pid=46, ppid=38, state=RUNNABLE:ACQUIRE_SPLIT_WAL_WORKER, hasLock=false; SplitWALProcedure 8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525}] 2023-10-15T16:04:40,281 DEBUG [PEWorker-2 {}] master.SplitWALManager(158): Acquired split WAL worker=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:40,281 DEBUG [PEWorker-4 {}] master.SplitWALManager(158): Acquired split WAL worker=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:40,282 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE, hasLock=false; SplitWALRemoteProcedure 8c5d72df62de%2C45999%2C1697385859285.1697385867224, worker=8c5d72df62de,46527,1697385859222}] 2023-10-15T16:04:40,282 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=48, ppid=46, state=RUNNABLE, hasLock=false; SplitWALRemoteProcedure 8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525, worker=8c5d72df62de,46527,1697385859222}] 2023-10-15T16:04:40,355 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:40,360 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:40,435 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46527 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.regionserver.SplitWALCallable, pid=48 2023-10-15T16:04:40,435 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46527 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.regionserver.SplitWALCallable, pid=47 2023-10-15T16:04:40,450 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] wal.WALSplitter(299): Splitting hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224, size=35.3 K (36157bytes) 2023-10-15T16:04:40,450 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224 2023-10-15T16:04:40,450 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] wal.WALSplitter(299): Splitting hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525, size=1.2 K (1261bytes) 2023-10-15T16:04:40,450 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525 2023-10-15T16:04:40,451 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224 after 1ms 2023-10-15T16:04:40,451 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525 after 1ms 2023-10-15T16:04:40,453 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] wal.WALSplitter(310): Open hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525 took 3ms 2023-10-15T16:04:40,454 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] wal.WALSplitter(310): Open hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224 took 4ms 2023-10-15T16:04:40,456 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] wal.WALSplitter(352): Last flushed sequenceid for 2d525cde81451d9cc8ed5e98268bcae5: last_flushed_sequence_id: 18446744073709551615 store_sequence_id { family_name: "hfileref" sequence_id: 7 } store_sequence_id { family_name: "queue" sequence_id: 19 } store_sequence_id { family_name: "sid" sequence_id: 7 } 2023-10-15T16:04:40,456 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] wal.WALSplitter(406): Finishing writing output for hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525 so closing down 2023-10-15T16:04:40,456 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] wal.OutputSink(108): Waiting for split writer threads to finish 2023-10-15T16:04:40,457 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0-Writer-0 {}] monitor.StreamSlowMonitor(122): New stream slow monitor 0000000000000000020-8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525.temp 2023-10-15T16:04:40,460 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0-Writer-0 {}] wal.AbstractRecoveredEditsOutputSink(71): Creating recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/0000000000000000020-8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525.temp 2023-10-15T16:04:40,460 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] wal.OutputSink(128): 3 split writer threads finished 2023-10-15T16:04:40,464 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] wal.WALSplitter(352): Last flushed sequenceid for f3bcdec0351888cb7677be57c1f453c1: last_flushed_sequence_id: 18446744073709551615 store_sequence_id { family_name: "cf" sequence_id: 3 } 2023-10-15T16:04:40,464 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] wal.WALSplitter(406): Finishing writing output for hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224 so closing down 2023-10-15T16:04:40,464 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] wal.OutputSink(108): Waiting for split writer threads to finish 2023-10-15T16:04:40,466 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1-Writer-0 {}] monitor.StreamSlowMonitor(122): New stream slow monitor 0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.1697385867224.temp 2023-10-15T16:04:40,467 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1-Writer-0 {}] wal.AbstractRecoveredEditsOutputSink(71): Creating recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.1697385867224.temp 2023-10-15T16:04:40,468 INFO [split-log-closeStream-pool-0 {}] wal.AbstractRecoveredEditsOutputSink(90): Closed recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/0000000000000000020-8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525.temp (wrote 3 edits, skipped 0 edits in 0 ms) 2023-10-15T16:04:40,469 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] wal.OutputSink(128): 3 split writer threads finished 2023-10-15T16:04:40,469 INFO [split-log-closeStream-pool-0 {}] wal.AbstractRecoveredEditsOutputSink(123): Rename recovered edits hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/0000000000000000020-8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525.temp to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/0000000000000000022 2023-10-15T16:04:40,469 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] wal.WALSplitter(425): Processed 4 edits across 1 Regions in 16 ms; skipped=1; WAL=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525, size=1.2 K, length=1261, corrupted=false, cancelled=false 2023-10-15T16:04:40,469 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] wal.WALSplitter(428): Completed split of hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525, journal: Splitting hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525, size=1.2 K (1261bytes) at 1697385880450Finishing writing output for hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525 so closing down at 1697385880456 (+6 ms)Creating recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/0000000000000000020-8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525.temp at 1697385880460 (+4 ms)3 split writer threads finished at 1697385880460Closed recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/0000000000000000020-8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525.temp (wrote 3 edits, skipped 0 edits in 0 ms) at 1697385880468 (+8 ms)Rename recovered edits hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/0000000000000000020-8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525.temp to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/0000000000000000022 at 1697385880469 (+1 ms)Processed 4 edits across 1 Regions in 16 ms; skipped=1; WAL=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525, size=1.2 K, length=1261, corrupted=false, cancelled=false at 1697385880469 2023-10-15T16:04:40,469 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] regionserver.SplitLogWorker(218): Done splitting WAL hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525 2023-10-15T16:04:40,469 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-0 {event_type=RS_LOG_REPLAY, pid=48}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=48 2023-10-15T16:04:40,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=48 2023-10-15T16:04:40,471 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] wal.WALSplitUtil(143): Moved hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525 to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs 2023-10-15T16:04:40,481 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=48, resume processing ppid=46 2023-10-15T16:04:40,481 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=48, ppid=46, state=SUCCESS, hasLock=false; SplitWALRemoteProcedure 8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525, worker=8c5d72df62de,46527,1697385859222 in 197 msec 2023-10-15T16:04:40,482 DEBUG [PEWorker-5 {}] master.SplitWALManager(172): Release split WAL worker=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:40,483 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1413): Finished pid=46, ppid=38, state=SUCCESS, hasLock=false; SplitWALProcedure 8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525, worker=8c5d72df62de,46527,1697385859222 in 202 msec 2023-10-15T16:04:40,490 INFO [split-log-closeStream-pool-0 {}] wal.AbstractRecoveredEditsOutputSink(90): Closed recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.1697385867224.temp (wrote 1 edits, skipped 0 edits in 1 ms) 2023-10-15T16:04:40,492 INFO [split-log-closeStream-pool-0 {}] wal.AbstractRecoveredEditsOutputSink(123): Rename recovered edits hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.1697385867224.temp to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/0000000000000000004 2023-10-15T16:04:40,492 INFO [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] wal.WALSplitter(425): Processed 1 edits across 1 Regions in 32 ms; skipped=0; WAL=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224, size=35.3 K, length=36157, corrupted=false, cancelled=false 2023-10-15T16:04:40,492 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] wal.WALSplitter(428): Completed split of hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224, journal: Splitting hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224, size=35.3 K (36157bytes) at 1697385880450Finishing writing output for hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224 so closing down at 1697385880464 (+14 ms)Creating recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.1697385867224.temp at 1697385880467 (+3 ms)3 split writer threads finished at 1697385880469 (+2 ms)Closed recovered edits writer path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.1697385867224.temp (wrote 1 edits, skipped 0 edits in 1 ms) at 1697385880490 (+21 ms)Rename recovered edits hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/0000000000000000004-8c5d72df62de%2C45999%2C1697385859285.1697385867224.temp to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/0000000000000000004 at 1697385880492 (+2 ms)Processed 1 edits across 1 Regions in 32 ms; skipped=0; WAL=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224, size=35.3 K, length=36157, corrupted=false, cancelled=false at 1697385880492 2023-10-15T16:04:40,492 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] regionserver.SplitLogWorker(218): Done splitting WAL hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224 2023-10-15T16:04:40,492 DEBUG [RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0-1 {event_type=RS_LOG_REPLAY, pid=47}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=47 2023-10-15T16:04:40,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=47 2023-10-15T16:04:40,495 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] wal.WALSplitUtil(143): Moved hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting/8c5d72df62de%2C45999%2C1697385859285.1697385867224 to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs 2023-10-15T16:04:40,498 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=47, resume processing ppid=45 2023-10-15T16:04:40,498 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=47, ppid=45, state=SUCCESS, hasLock=false; SplitWALRemoteProcedure 8c5d72df62de%2C45999%2C1697385859285.1697385867224, worker=8c5d72df62de,46527,1697385859222 in 214 msec 2023-10-15T16:04:40,498 DEBUG [PEWorker-2 {}] master.SplitWALManager(172): Release split WAL worker=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:40,500 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=45, resume processing ppid=38 2023-10-15T16:04:40,500 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1413): Finished pid=45, ppid=38, state=SUCCESS, hasLock=false; SplitWALProcedure 8c5d72df62de%2C45999%2C1697385859285.1697385867224, worker=8c5d72df62de,46527,1697385859222 in 219 msec 2023-10-15T16:04:40,502 INFO [PEWorker-1 {}] master.SplitLogManager(171): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,45999,1697385859285-splitting dir is empty, no logs to split. 2023-10-15T16:04:40,502 INFO [PEWorker-1 {}] master.SplitWALManager(105): 8c5d72df62de,45999,1697385859285 WAL count=0, meta=false 2023-10-15T16:04:40,502 DEBUG [PEWorker-1 {}] procedure.ServerCrashProcedure(323): Check if 8c5d72df62de,45999,1697385859285 WAL splitting is done? wals=0, meta=false 2023-10-15T16:04:40,503 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=49, ppid=38, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN}, {pid=50, ppid=38, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=SyncRep, region=f3bcdec0351888cb7677be57c1f453c1, ASSIGN}] 2023-10-15T16:04:40,504 INFO [PEWorker-1 {}] procedure.MasterProcedureScheduler(792): Took xlock for pid=50, ppid=38, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=SyncRep, region=f3bcdec0351888cb7677be57c1f453c1, ASSIGN 2023-10-15T16:04:40,504 INFO [PEWorker-3 {}] procedure.MasterProcedureScheduler(792): Took xlock for pid=49, ppid=38, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN 2023-10-15T16:04:40,505 INFO [PEWorker-3 {}] assignment.TransitRegionStateProcedure(262): Starting pid=49, ppid=38, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-10-15T16:04:40,505 INFO [PEWorker-1 {}] assignment.TransitRegionStateProcedure(262): Starting pid=50, ppid=38, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true; TransitRegionStateProcedure table=SyncRep, region=f3bcdec0351888cb7677be57c1f453c1, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-10-15T16:04:40,655 DEBUG [8c5d72df62de:33403 {}] assignment.AssignmentManager(2316): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-10-15T16:04:40,655 INFO [PEWorker-4 {}] assignment.RegionStateStore(223): pid=49 updating hbase:meta row=2d525cde81451d9cc8ed5e98268bcae5, regionState=OPENING, regionLocation=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:40,655 INFO [PEWorker-2 {}] assignment.RegionStateStore(223): pid=50 updating hbase:meta row=f3bcdec0351888cb7677be57c1f453c1, regionState=OPENING, regionLocation=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:40,656 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:45999 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:45999 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:40,657 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,45999,1697385859285, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,45999,1697385859285, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:45999 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:45999 2023-10-15T16:04:40,657 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,45999,1697385859285, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:04:40,657 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,45999,1697385859285, seqNum=-1 from cache 2023-10-15T16:04:40,658 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,45999,1697385859285, seqNum=-1 , the old value is null, error=java.net.ConnectException: Call to address=8c5d72df62de:45999 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:45999 2023-10-15T16:04:40,658 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:45999 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:45999 2023-10-15T16:04:40,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:04:40,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,46527,1697385859222, seqNum=-1] 2023-10-15T16:04:40,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.RpcConnection(127): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-15T16:04:40,762 INFO [MiniHBaseClusterRegionServer-EventLoopGroup-11-3 {}] ipc.ServerRpcConnection(478): Connection from 172.17.0.3:32897, version=4.0.0-alpha-1-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-15T16:04:40,763 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=51, ppid=49, state=RUNNABLE, hasLock=false; OpenRegionProcedure 2d525cde81451d9cc8ed5e98268bcae5, server=8c5d72df62de,46527,1697385859222}] 2023-10-15T16:04:40,764 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=52, ppid=50, state=RUNNABLE, hasLock=false; OpenRegionProcedure f3bcdec0351888cb7677be57c1f453c1, server=8c5d72df62de,46527,1697385859222}] 2023-10-15T16:04:40,897 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:40,918 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C46527%2C1697385859222]: currently replicating from: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 at position: -1 2023-10-15T16:04:40,927 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] handler.AssignRegionHandler(127): Open hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:40,928 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] wal.WALFactory(196): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-10-15T16:04:40,928 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] monitor.StreamSlowMonitor(122): New stream slow monitor rep 2023-10-15T16:04:40,932 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] wal.AbstractFSWAL(602): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=8c5d72df62de%2C46527%2C1697385859222.rep, suffix=, logDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222, archiveDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs, maxLogs=10 2023-10-15T16:04:40,932 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] handler.AssignRegionHandler(127): Open SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:40,933 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(7727): Opening region: {ENCODED => f3bcdec0351888cb7677be57c1f453c1, NAME => 'SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1.', STARTKEY => '', ENDKEY => ''} 2023-10-15T16:04:40,933 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table SyncRep f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:40,933 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(891): Instantiated SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=0 ; minColumnNum=100 ; preparePutThreadLimit=0 ; hotProtect now disable 2023-10-15T16:04:40,933 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(7769): checking encryption for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:40,933 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(7772): checking classloading for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:40,937 INFO [StoreOpener-f3bcdec0351888cb7677be57c1f453c1-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:40,939 INFO [StoreOpener-f3bcdec0351888cb7677be57c1f453c1-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f3bcdec0351888cb7677be57c1f453c1 columnFamilyName cf 2023-10-15T16:04:40,939 DEBUG [StoreOpener-f3bcdec0351888cb7677be57c1f453c1-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:40,939 INFO [StoreOpener-f3bcdec0351888cb7677be57c1f453c1-1 {}] regionserver.HStore(324): Store=f3bcdec0351888cb7677be57c1f453c1/cf, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:40,940 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(1028): replaying wal for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:40,941 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:40,944 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(5511): Found 1 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:40,948 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(5578): Replaying edits from hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/0000000000000000004 2023-10-15T16:04:40,954 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.rep.1697385880933, exclude list is [], retry=0 2023-10-15T16:04:40,959 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(5769): Applied 1000, skipped 0, firstSequenceIdInLog=4, maxSequenceIdInLog=4, path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/0000000000000000004 2023-10-15T16:04:40,959 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(2885): Flushing f3bcdec0351888cb7677be57c1f453c1 1/1 column families, dataSize=31.25 KB heapSize=101.81 KB 2023-10-15T16:04:40,961 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:40,962 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:40,962 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:40,986 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/.tmp/cf/9e5ff1a6ebb340fe803f1a85f55dc61b is 36, key is \x00\x00\x00\x00/cf:cq/1697385870586/Put/seqid=0 2023-10-15T16:04:40,993 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] wal.AbstractFSWAL(980): New WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.rep.1697385880933 2023-10-15T16:04:40,993 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK], DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK], DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK]] 2023-10-15T16:04:40,994 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(7727): Opening region: {ENCODED => 2d525cde81451d9cc8ed5e98268bcae5, NAME => 'hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.', STARTKEY => '', ENDKEY => ''} 2023-10-15T16:04:40,994 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-10-15T16:04:40,994 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(8255): Registered coprocessor service: region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. service=MultiRowMutationService 2023-10-15T16:04:40,994 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.RegionCoprocessorHost(391): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:replication successfully. 2023-10-15T16:04:40,994 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table replication 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:40,994 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(891): Instantiated hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=0 ; minColumnNum=100 ; preparePutThreadLimit=0 ; hotProtect now disable 2023-10-15T16:04:40,994 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(7769): checking encryption for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:40,994 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(7772): checking classloading for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:40,996 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family hfileref of region 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:40,997 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d525cde81451d9cc8ed5e98268bcae5 columnFamilyName hfileref 2023-10-15T16:04:40,997 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:41,015 INFO [StoreFileOpener-2d525cde81451d9cc8ed5e98268bcae5-hfileref-1 {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 06c1db2ad5514d38945c79bef8c56cc7 2023-10-15T16:04:41,015 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/hfileref/06c1db2ad5514d38945c79bef8c56cc7 2023-10-15T16:04:41,015 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(324): Store=2d525cde81451d9cc8ed5e98268bcae5/hfileref, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:41,015 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family queue of region 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:41,016 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d525cde81451d9cc8ed5e98268bcae5 columnFamilyName queue 2023-10-15T16:04:41,016 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:41,022 INFO [StoreFileOpener-2d525cde81451d9cc8ed5e98268bcae5-queue-1 {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46fdfd784a1242dd9339a7a4fc97d13a 2023-10-15T16:04:41,022 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/46fdfd784a1242dd9339a7a4fc97d13a 2023-10-15T16:04:41,023 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(324): Store=2d525cde81451d9cc8ed5e98268bcae5/queue, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:41,023 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family sid of region 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:41,024 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d525cde81451d9cc8ed5e98268bcae5 columnFamilyName sid 2023-10-15T16:04:41,024 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:41,027 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=31.25 KB at sequenceid=4 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/.tmp/cf/9e5ff1a6ebb340fe803f1a85f55dc61b 2023-10-15T16:04:41,030 INFO [StoreFileOpener-2d525cde81451d9cc8ed5e98268bcae5-sid-1 {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 748fb97f697941489dd7ef4042a092d8 2023-10-15T16:04:41,030 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/sid/748fb97f697941489dd7ef4042a092d8 2023-10-15T16:04:41,030 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(324): Store=2d525cde81451d9cc8ed5e98268bcae5/sid, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:41,030 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(1028): replaying wal for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:41,031 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:41,032 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(5511): Found 1 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:41,033 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/.tmp/cf/9e5ff1a6ebb340fe803f1a85f55dc61b as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/cf/9e5ff1a6ebb340fe803f1a85f55dc61b 2023-10-15T16:04:41,033 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(5578): Replaying edits from hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/0000000000000000022 2023-10-15T16:04:41,036 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(5769): Applied 4, skipped 0, firstSequenceIdInLog=20, maxSequenceIdInLog=22, path=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/0000000000000000022 2023-10-15T16:04:41,036 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(2885): Flushing 2d525cde81451d9cc8ed5e98268bcae5 3/3 column families, dataSize=521 B heapSize=1.55 KB 2023-10-15T16:04:41,039 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/cf/9e5ff1a6ebb340fe803f1a85f55dc61b, entries=1000, sequenceid=4, filesize=39.0 K 2023-10-15T16:04:41,040 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(3123): Finished flush of dataSize ~31.25 KB/32000, heapSize ~101.80 KB/104240, currentSize=0 B/0 for f3bcdec0351888cb7677be57c1f453c1 in 80ms, sequenceid=4, compaction requested=false; wal=null 2023-10-15T16:04:41,040 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.MetricsTableSourceImpl(133): Creating new MetricsTableSourceImpl for table 'SyncRep' 2023-10-15T16:04:41,041 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(5388): Deleted recovered.edits file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/0000000000000000004 2023-10-15T16:04:41,044 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(1038): stopping wal replay for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:41,044 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(1050): Cleaning up temporary data for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:41,047 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(1083): writing seq id for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:41,051 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] wal.WALSplitUtil(410): Wrote file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-10-15T16:04:41,052 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(1104): Opened f3bcdec0351888cb7677be57c1f453c1; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73700924, jitterRate=0.09822934865951538}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-15T16:04:41,052 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(1112): Running coprocessor post-open hooks for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:41,052 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegion(996): Region open journal for f3bcdec0351888cb7677be57c1f453c1: Running coprocessor pre-open hook at 1697385880933Writing region info on filesystem at 1697385880933Initializing all the Stores at 1697385880936 (+3 ms)Instantiating store for column family {NAME => 'cf', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385880936Obtaining lock to block concurrent updates at 1697385880959 (+23 ms)Preparing flush snapshotting stores in f3bcdec0351888cb7677be57c1f453c1 at 1697385880959Finished memstore snapshotting SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1., syncing WAL and waiting on mvcc, flushsize=dataSize=32000, getHeapSize=104240, getOffHeapSize=0, getCellsCount=1000 at 1697385880959Flushing stores of SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. at 1697385880959Flushing f3bcdec0351888cb7677be57c1f453c1/cf: creating writer at 1697385880959Flushing f3bcdec0351888cb7677be57c1f453c1/cf: appending metadata at 1697385880986 (+27 ms)Flushing f3bcdec0351888cb7677be57c1f453c1/cf: closing flushed file at 1697385880986Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@4c464b88: reopening flushed file at 1697385881032 (+46 ms)Finished flush of dataSize ~31.25 KB/32000, heapSize ~101.80 KB/104240, currentSize=0 B/0 for f3bcdec0351888cb7677be57c1f453c1 in 80ms, sequenceid=4, compaction requested=false; wal=null at 1697385881040 (+8 ms)Cleaning up temporary data from old regions at 1697385881044 (+4 ms)Running coprocessor post-open hooks at 1697385881052 (+8 ms)Region opened successfully at 1697385881052 2023-10-15T16:04:41,053 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegionServer(2186): Post open deploy tasks for SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1., pid=52, masterSystemTime=1697385880916 2023-10-15T16:04:41,057 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] regionserver.HRegionServer(2216): Finished post open deploy task for SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:41,057 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=52}] handler.AssignRegionHandler(147): Opened SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:41,057 INFO [PEWorker-3 {}] assignment.RegionStateStore(223): pid=50 updating hbase:meta row=f3bcdec0351888cb7677be57c1f453c1, regionState=OPEN, repBarrier=5, openSeqNum=5, regionLocation=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:41,059 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/queue/8807426219fd436c817c02ff77937323 is 193, key is 1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175/queue:8c5d72df62de%2C38435%2C1697385859175.rep/1697385879027/Put/seqid=0 2023-10-15T16:04:41,062 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=52, resume processing ppid=50 2023-10-15T16:04:41,062 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=52, ppid=50, state=SUCCESS, hasLock=false; OpenRegionProcedure f3bcdec0351888cb7677be57c1f453c1, server=8c5d72df62de,46527,1697385859222 in 295 msec 2023-10-15T16:04:41,063 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=50, ppid=38, state=SUCCESS, hasLock=false; TransitRegionStateProcedure table=SyncRep, region=f3bcdec0351888cb7677be57c1f453c1, ASSIGN in 559 msec 2023-10-15T16:04:41,071 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=521 B at sequenceid=22 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/queue/8807426219fd436c817c02ff77937323 2023-10-15T16:04:41,077 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8807426219fd436c817c02ff77937323 2023-10-15T16:04:41,078 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/queue/8807426219fd436c817c02ff77937323 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/8807426219fd436c817c02ff77937323 2023-10-15T16:04:41,085 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8807426219fd436c817c02ff77937323 2023-10-15T16:04:41,085 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/8807426219fd436c817c02ff77937323, entries=4, sequenceid=22, filesize=5.8 K 2023-10-15T16:04:41,085 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(3123): Finished flush of dataSize ~521 B/521, heapSize ~1.04 KB/1064, currentSize=0 B/0 for 2d525cde81451d9cc8ed5e98268bcae5 in 49ms, sequenceid=22, compaction requested=false; wal=null 2023-10-15T16:04:41,085 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.MetricsTableSourceImpl(133): Creating new MetricsTableSourceImpl for table 'hbase:replication' 2023-10-15T16:04:41,086 DEBUG [HBase-Metrics2-1 {}] impl.GlobalMetricRegistriesAdapter(153): Removing adapter for the MetricRegistry: RegionServer,sub=TableRequests_Namespace_hbase_table_replication 2023-10-15T16:04:41,087 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(5388): Deleted recovered.edits file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/0000000000000000022 2023-10-15T16:04:41,088 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(1038): stopping wal replay for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:41,088 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(1050): Cleaning up temporary data for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:41,089 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:replication descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-10-15T16:04:41,091 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(1083): writing seq id for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:41,093 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] wal.WALSplitUtil(410): Wrote file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/22.seqid, newMaxSeqId=22, maxSeqId=17 2023-10-15T16:04:41,094 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(1104): Opened 2d525cde81451d9cc8ed5e98268bcae5; next sequenceid=23; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75142684, jitterRate=0.11971324682235718}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-10-15T16:04:41,094 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(1112): Running coprocessor post-open hooks for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:41,095 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegion(996): Region open journal for 2d525cde81451d9cc8ed5e98268bcae5: Running coprocessor pre-open hook at 1697385880994Writing region info on filesystem at 1697385880995 (+1 ms)Initializing all the Stores at 1697385880996 (+1 ms)Instantiating store for column family {NAME => 'hfileref', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385880996Instantiating store for column family {NAME => 'queue', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385880996Instantiating store for column family {NAME => 'sid', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385880996Obtaining lock to block concurrent updates at 1697385881036 (+40 ms)Preparing flush snapshotting stores in 2d525cde81451d9cc8ed5e98268bcae5 at 1697385881036Finished memstore snapshotting hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., syncing WAL and waiting on mvcc, flushsize=dataSize=521, getHeapSize=1544, getOffHeapSize=0, getCellsCount=4 at 1697385881036Flushing stores of hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. at 1697385881036Flushing 2d525cde81451d9cc8ed5e98268bcae5/queue: creating writer at 1697385881036Flushing 2d525cde81451d9cc8ed5e98268bcae5/queue: appending metadata at 1697385881057 (+21 ms)Flushing 2d525cde81451d9cc8ed5e98268bcae5/queue: closing flushed file at 1697385881057Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@1984b6aa: reopening flushed file at 1697385881077 (+20 ms)Finished flush of dataSize ~521 B/521, heapSize ~1.04 KB/1064, currentSize=0 B/0 for 2d525cde81451d9cc8ed5e98268bcae5 in 49ms, sequenceid=22, compaction requested=false; wal=null at 1697385881085 (+8 ms)Cleaning up temporary data from old regions at 1697385881088 (+3 ms)Running coprocessor post-open hooks at 1697385881094 (+6 ms)Region opened successfully at 1697385881095 (+1 ms) 2023-10-15T16:04:41,097 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegionServer(2186): Post open deploy tasks for hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., pid=51, masterSystemTime=1697385880916 2023-10-15T16:04:41,099 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] regionserver.HRegionServer(2216): Finished post open deploy task for hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:41,099 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=51}] handler.AssignRegionHandler(147): Opened hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:41,105 INFO [PEWorker-2 {}] assignment.RegionStateStore(223): pid=49 updating hbase:meta row=2d525cde81451d9cc8ed5e98268bcae5, regionState=OPEN, openSeqNum=23, regionLocation=8c5d72df62de,46527,1697385859222 2023-10-15T16:04:41,109 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=51, resume processing ppid=49 2023-10-15T16:04:41,110 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1413): Finished pid=51, ppid=49, state=SUCCESS, hasLock=false; OpenRegionProcedure 2d525cde81451d9cc8ed5e98268bcae5, server=8c5d72df62de,46527,1697385859222 in 344 msec 2023-10-15T16:04:41,112 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=49, resume processing ppid=38 2023-10-15T16:04:41,112 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=49, ppid=38, state=SUCCESS, hasLock=false; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN in 607 msec 2023-10-15T16:04:41,113 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=53, ppid=38, state=RUNNABLE:ASSIGN_REPLICATION_QUEUES_ADD_MISSING_QUEUES, hasLock=false; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure}] 2023-10-15T16:04:41,116 DEBUG [PEWorker-3 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:45999 this server is in the failed servers list 2023-10-15T16:04:41,116 DEBUG [PEWorker-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,45999,1697385859285, seqNum=18 , the old value is region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,45999,1697385859285, seqNum=18, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:45999 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:45999 2023-10-15T16:04:41,116 DEBUG [PEWorker-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,45999,1697385859285, seqNum=18 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:45999 2023-10-15T16:04:41,116 DEBUG [PEWorker-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,45999,1697385859285, seqNum=18 from cache 2023-10-15T16:04:41,234 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'hbase:replication', row='', locateType=CURRENT is [region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,46527,1697385859222, seqNum=23] 2023-10-15T16:04:41,292 DEBUG [PEWorker-3 {}] replication.AssignReplicationQueuesProcedure(120): There are 2 replication queues need to be claimed for 8c5d72df62de,45999,1697385859285 2023-10-15T16:04:41,292 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.ClaimReplicationQueueRemoteProcedure}] 2023-10-15T16:04:41,364 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 0, reset compression=false 2023-10-15T16:04:41,372 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=0, beingWritten=true 2023-10-15T16:04:41,378 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceWALReader(175): Read 0 WAL entries eligible for replication 2023-10-15T16:04:41,378 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:41,378 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,46527,1697385859222 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, lastWalPosition=383, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-10-15T16:04:41,379 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:38435 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:38435 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:41,380 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2 , the old value is region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2, error=java.net.ConnectException: Call to address=8c5d72df62de:38435 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:38435 2023-10-15T16:04:41,380 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:04:41,380 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,38435,1697385859175, seqNum=2 from cache 2023-10-15T16:04:41,381 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:38435 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:38435 2023-10-15T16:04:41,388 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:41,452 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46527 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.ClaimReplicationQueueCallable, pid=54 2023-10-15T16:04:41,491 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:45999 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:45999 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:41,491 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,45999,1697385859285, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,45999,1697385859285, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:45999 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:45999 2023-10-15T16:04:41,492 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,45999,1697385859285, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:04:41,492 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,45999,1697385859285, seqNum=-1 from cache 2023-10-15T16:04:41,492 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:45999 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:45999 2023-10-15T16:04:41,593 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:41,596 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:41,600 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:04:41,600 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,46527,1697385859222, seqNum=-1] 2023-10-15T16:04:41,601 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.RpcConnection(127): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-15T16:04:41,602 INFO [MiniHBaseClusterRegionServer-EventLoopGroup-11-1 {}] ipc.ServerRpcConnection(478): Connection from 172.17.0.3:45081, version=4.0.0-alpha-1-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=ClientService 2023-10-15T16:04:41,606 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'hbase:replication', row='1-8c5d72df62de,46527,1697385859222', locateType=CURRENT is [region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,46527,1697385859222, seqNum=23] 2023-10-15T16:04:41,897 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:41,900 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:41,905 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:42,308 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:42,312 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:42,817 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:42,824 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:42,898 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:43,430 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:43,435 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:43,898 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:44,140 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:44,144 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:44,525 WARN [HBase-Metrics2-1 {}] impl.MetricsConfig(136): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-10-15T16:04:44,898 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:44,899 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=54}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:44,901 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=54}] wal.AbstractFSWALProvider(366): Seprated old WAL dir hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de,45999,1697385859285 not exists 2023-10-15T16:04:44,901 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=54}] regionserver.ReplicationSourceManager(843): Skip enqueuing log 8c5d72df62de%2C45999%2C1697385859285.1697385861037 because it is before the start offset 8c5d72df62de%2C45999%2C1697385859285.1697385861037:-1 2023-10-15T16:04:44,902 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=54}] regionserver.ReplicationSourceManager(843): Skip enqueuing log 8c5d72df62de%2C45999%2C1697385859285.meta.1697385861367.meta because it is before the start offset null 2023-10-15T16:04:44,902 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=54}] regionserver.ReplicationSourceManager(843): Skip enqueuing log 8c5d72df62de%2C45999%2C1697385859285.meta.1697385867283.meta because it is before the start offset null 2023-10-15T16:04:44,902 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=54}] regionserver.ReplicationSourceManager(888): Added source for recovered queue 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285, number of wals to replicate: 2 2023-10-15T16:04:44,902 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=54}] regionserver.ReplicationSourceManager(891): Enqueueing log hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C45999%2C1697385859285.1697385867224 from recovered queue for source: 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 2023-10-15T16:04:44,902 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=54}] regionserver.ReplicationSourceManager(891): Enqueueing log hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525 from recovered queue for source: 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 2023-10-15T16:04:44,902 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=54}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=54 2023-10-15T16:04:44,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=54 2023-10-15T16:04:44,907 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=54, resume processing ppid=53 2023-10-15T16:04:44,907 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1413): Finished pid=54, ppid=53, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.ClaimReplicationQueueRemoteProcedure in 3.6130 sec 2023-10-15T16:04:44,916 DEBUG [PEWorker-2 {}] replication.AssignReplicationQueuesProcedure(120): There are 1 replication queues need to be claimed for 8c5d72df62de,45999,1697385859285 2023-10-15T16:04:44,916 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=55, ppid=53, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.ClaimReplicationQueueRemoteProcedure}] 2023-10-15T16:04:44,919 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:44,926 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x57db6dea to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:44,927 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:44,928 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f51002c connected 2023-10-15T16:04:44,936 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@59277de5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:44,937 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 (queues=2) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:44,937 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C45999%2C1697385859285 2023-10-15T16:04:44,937 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:44,940 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C45999%2C1697385859285.1697385867224, startPosition=0, beingWritten=false 2023-10-15T16:04:44,940 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C45999%2C1697385859285.rep 2023-10-15T16:04:44,940 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C45999%2C1697385859285 2023-10-15T16:04:44,940 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:44,941 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525, startPosition=0, beingWritten=false 2023-10-15T16:04:44,945 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(451): EOF, closing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C45999%2C1697385859285.1697385867224 2023-10-15T16:04:44,945 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceWALReader(175): Read 1 WAL entries eligible for replication 2023-10-15T16:04:44,945 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(236): No more WAL files in queue 2023-10-15T16:04:44,945 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceWALReader(118): Stopping the replication source wal reader 2023-10-15T16:04:44,946 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 got entry batch from reader: WALEntryBatch [walEntries=[{SyncRep/f3bcdec0351888cb7677be57c1f453c1/4=[#edits: 1000 = <\x00\x00\x00\x00/cf:cq/1697385870586/Put/vlen=4/seqid=0; \x00\x00\x00\x01/cf:cq/1697385870592/Put/vlen=4/seqid=0; \x00\x00\x00\x02/cf:cq/1697385870595/Put/vlen=4/seqid=0; \x00\x00\x00\x03/cf:cq/1697385870598/Put/vlen=4/seqid=0; \x00\x00\x00\x04/cf:cq/1697385870600/Put/vlen=4/seqid=0; \x00\x00\x00\x05/cf:cq/1697385870602/Put/vlen=4/seqid=0; \x00\x00\x00\x06/cf:cq/1697385870604/Put/vlen=4/seqid=0; \x00\x00\x00\x07/cf:cq/1697385870607/Put/vlen=4/seqid=0; \x00\x00\x00\x08/cf:cq/1697385870609/Put/vlen=4/seqid=0; \x00\x00\x00\x09/cf:cq/1697385870611/Put/vlen=4/seqid=0; \x00\x00\x00\x0A/cf:cq/1697385870613/Put/vlen=4/seqid=0; \x00\x00\x00\x0B/cf:cq/1697385870615/Put/vlen=4/seqid=0; \x00\x00\x00\x0C/cf:cq/1697385870617/Put/vlen=4/seqid=0; \x00\x00\x00\x0D/cf:cq/1697385870619/Put/vlen=4/seqid=0; \x00\x00\x00\x0E/cf:cq/1697385870621/Put/vlen=4/seqid=0; \x00\x00\x00\x0F/cf:cq/1697385870623/Put/vlen=4/seqid=0; \x00\x00\x00\x10/cf:cq/1697385870625/Put/vlen=4/seqid=0; \x00\x00\x00\x11/cf:cq/1697385870627/Put/vlen=4/seqid=0; \x00\x00\x00\x12/cf:cq/1697385870629/Put/vlen=4/seqid=0; \x00\x00\x00\x13/cf:cq/1697385870631/Put/vlen=4/seqid=0; \x00\x00\x00\x14/cf:cq/1697385870633/Put/vlen=4/seqid=0; \x00\x00\x00\x15/cf:cq/1697385870635/Put/vlen=4/seqid=0; \x00\x00\x00\x16/cf:cq/1697385870638/Put/vlen=4/seqid=0; \x00\x00\x00\x17/cf:cq/1697385870639/Put/vlen=4/seqid=0; \x00\x00\x00\x18/cf:cq/1697385870641/Put/vlen=4/seqid=0; \x00\x00\x00\x19/cf:cq/1697385870643/Put/vlen=4/seqid=0; \x00\x00\x00\x1A/cf:cq/1697385870645/Put/vlen=4/seqid=0; \x00\x00\x00\x1B/cf:cq/1697385870647/Put/vlen=4/seqid=0; \x00\x00\x00\x1C/cf:cq/1697385870649/Put/vlen=4/seqid=0; \x00\x00\x00\x1D/cf:cq/1697385870651/Put/vlen=4/seqid=0; \x00\x00\x00\x1E/cf:cq/1697385870653/Put/vlen=4/seqid=0; \x00\x00\x00\x1F/cf:cq/1697385870655/Put/vlen=4/seqid=0; \x00\x00\x00 /cf:cq/1697385870657/Put/vlen=4/seqid=0; \x00\x00\x00!/cf:cq/1697385870659/Put/vlen=4/seqid=0; \x00\x00\x00"/cf:cq/1697385870662/Put/vlen=4/seqid=0; \x00\x00\x00#/cf:cq/1697385870664/Put/vlen=4/seqid=0; \x00\x00\x00$/cf:cq/1697385870666/Put/vlen=4/seqid=0; \x00\x00\x00%/cf:cq/1697385870668/Put/vlen=4/seqid=0; \x00\x00\x00&/cf:cq/1697385870671/Put/vlen=4/seqid=0; \x00\x00\x00'/cf:cq/1697385870673/Put/vlen=4/seqid=0; \x00\x00\x00(/cf:cq/1697385870676/Put/vlen=4/seqid=0; \x00\x00\x00)/cf:cq/1697385870679/Put/vlen=4/seqid=0; \x00\x00\x00*/cf:cq/1697385870683/Put/vlen=4/seqid=0; \x00\x00\x00+/cf:cq/1697385870685/Put/vlen=4/seqid=0; \x00\x00\x00,/cf:cq/1697385870686/Put/vlen=4/seqid=0; \x00\x00\x00-/cf:cq/1697385870688/Put/vlen=4/seqid=0; \x00\x00\x00./cf:cq/1697385870690/Put/vlen=4/seqid=0; \x00\x00\x00//cf:cq/1697385870691/Put/vlen=4/seqid=0; \x00\x00\x000/cf:cq/1697385870693/Put/vlen=4/seqid=0; \x00\x00\x001/cf:cq/1697385870695/Put/vlen=4/seqid=0; \x00\x00\x002/cf:cq/1697385870697/Put/vlen=4/seqid=0; \x00\x00\x003/cf:cq/1697385870699/Put/vlen=4/seqid=0; \x00\x00\x004/cf:cq/1697385870701/Put/vlen=4/seqid=0; \x00\x00\x005/cf:cq/1697385870703/Put/vlen=4/seqid=0; \x00\x00\x006/cf:cq/1697385870704/Put/vlen=4/seqid=0; \x00\x00\x007/cf:cq/1697385870706/Put/vlen=4/seqid=0; \x00\x00\x008/cf:cq/1697385870708/Put/vlen=4/seqid=0; \x00\x00\x009/cf:cq/1697385870711/Put/vlen=4/seqid=0; \x00\x00\x00:/cf:cq/1697385870712/Put/vlen=4/seqid=0; \x00\x00\x00;/cf:cq/1697385870715/Put/vlen=4/seqid=0; \x00\x00\x00/cf:cq/1697385870720/Put/vlen=4/seqid=0; \x00\x00\x00?/cf:cq/1697385870721/Put/vlen=4/seqid=0; \x00\x00\x00@/cf:cq/1697385870723/Put/vlen=4/seqid=0; \x00\x00\x00A/cf:cq/1697385870725/Put/vlen=4/seqid=0; \x00\x00\x00B/cf:cq/1697385870727/Put/vlen=4/seqid=0; \x00\x00\x00C/cf:cq/1697385870729/Put/vlen=4/seqid=0; \x00\x00\x00D/cf:cq/1697385870731/Put/vlen=4/seqid=0; \x00\x00\x00E/cf:cq/1697385870733/Put/vlen=4/seqid=0; \x00\x00\x00F/cf:cq/1697385870734/Put/vlen=4/seqid=0; \x00\x00\x00G/cf:cq/1697385870736/Put/vlen=4/seqid=0; \x00\x00\x00H/cf:cq/1697385870738/Put/vlen=4/seqid=0; \x00\x00\x00I/cf:cq/1697385870740/Put/vlen=4/seqid=0; \x00\x00\x00J/cf:cq/1697385870742/Put/vlen=4/seqid=0; \x00\x00\x00K/cf:cq/1697385870743/Put/vlen=4/seqid=0; \x00\x00\x00L/cf:cq/1697385870745/Put/vlen=4/seqid=0; \x00\x00\x00M/cf:cq/1697385870747/Put/vlen=4/seqid=0; \x00\x00\x00N/cf:cq/1697385870749/Put/vlen=4/seqid=0; \x00\x00\x00O/cf:cq/1697385870751/Put/vlen=4/seqid=0; \x00\x00\x00P/cf:cq/1697385870753/Put/vlen=4/seqid=0; \x00\x00\x00Q/cf:cq/1697385870755/Put/vlen=4/seqid=0; \x00\x00\x00R/cf:cq/1697385870756/Put/vlen=4/seqid=0; \x00\x00\x00S/cf:cq/1697385870758/Put/vlen=4/seqid=0; \x00\x00\x00T/cf:cq/1697385870760/Put/vlen=4/seqid=0; \x00\x00\x00U/cf:cq/1697385870762/Put/vlen=4/seqid=0; \x00\x00\x00V/cf:cq/1697385870765/Put/vlen=4/seqid=0; \x00\x00\x00W/cf:cq/1697385870767/Put/vlen=4/seqid=0; \x00\x00\x00X/cf:cq/1697385870769/Put/vlen=4/seqid=0; \x00\x00\x00Y/cf:cq/1697385870771/Put/vlen=4/seqid=0; \x00\x00\x00Z/cf:cq/1697385870773/Put/vlen=4/seqid=0; \x00\x00\x00[/cf:cq/1697385870774/Put/vlen=4/seqid=0; \x00\x00\x00\x5C/cf:cq/1697385870776/Put/vlen=4/seqid=0; \x00\x00\x00]/cf:cq/1697385870778/Put/vlen=4/seqid=0; \x00\x00\x00^/cf:cq/1697385870780/Put/vlen=4/seqid=0; \x00\x00\x00_/cf:cq/1697385870783/Put/vlen=4/seqid=0; \x00\x00\x00`/cf:cq/1697385870785/Put/vlen=4/seqid=0; \x00\x00\x00a/cf:cq/1697385870786/Put/vlen=4/seqid=0; \x00\x00\x00b/cf:cq/1697385870788/Put/vlen=4/seqid=0; \x00\x00\x00c/cf:cq/1697385870789/Put/vlen=4/seqid=0; \x00\x00\x00d/cf:cq/1697385870791/Put/vlen=4/seqid=0; \x00\x00\x00e/cf:cq/1697385870795/Put/vlen=4/seqid=0; \x00\x00\x00f/cf:cq/1697385870797/Put/vlen=4/seqid=0; \x00\x00\x00g/cf:cq/1697385870798/Put/vlen=4/seqid=0; \x00\x00\x00h/cf:cq/1697385870800/Put/vlen=4/seqid=0; \x00\x00\x00i/cf:cq/1697385870801/Put/vlen=4/seqid=0; \x00\x00\x00j/cf:cq/1697385870803/Put/vlen=4/seqid=0; \x00\x00\x00k/cf:cq/1697385870805/Put/vlen=4/seqid=0; \x00\x00\x00l/cf:cq/1697385870806/Put/vlen=4/seqid=0; \x00\x00\x00m/cf:cq/1697385870808/Put/vlen=4/seqid=0; \x00\x00\x00n/cf:cq/1697385870810/Put/vlen=4/seqid=0; \x00\x00\x00o/cf:cq/1697385870812/Put/vlen=4/seqid=0; \x00\x00\x00p/cf:cq/1697385870814/Put/vlen=4/seqid=0; \x00\x00\x00q/cf:cq/1697385870816/Put/vlen=4/seqid=0; \x00\x00\x00r/cf:cq/1697385870818/Put/vlen=4/seqid=0; \x00\x00\x00s/cf:cq/1697385870820/Put/vlen=4/seqid=0; \x00\x00\x00t/cf:cq/1697385870822/Put/vlen=4/seqid=0; \x00\x00\x00u/cf:cq/1697385870824/Put/vlen=4/seqid=0; \x00\x00\x00v/cf:cq/1697385870825/Put/vlen=4/seqid=0; \x00\x00\x00w/cf:cq/1697385870827/Put/vlen=4/seqid=0; \x00\x00\x00x/cf:cq/1697385870829/Put/vlen=4/seqid=0; \x00\x00\x00y/cf:cq/1697385870830/Put/vlen=4/seqid=0; \x00\x00\x00z/cf:cq/1697385870832/Put/vlen=4/seqid=0; \x00\x00\x00{/cf:cq/1697385870834/Put/vlen=4/seqid=0; \x00\x00\x00|/cf:cq/1697385870835/Put/vlen=4/seqid=0; \x00\x00\x00}/cf:cq/1697385870837/Put/vlen=4/seqid=0; \x00\x00\x00~/cf:cq/1697385870839/Put/vlen=4/seqid=0; \x00\x00\x00\x7F/cf:cq/1697385870845/Put/vlen=4/seqid=0; \x00\x00\x00\x80/cf:cq/1697385870847/Put/vlen=4/seqid=0; \x00\x00\x00\x81/cf:cq/1697385870849/Put/vlen=4/seqid=0; \x00\x00\x00\x82/cf:cq/1697385870851/Put/vlen=4/seqid=0; \x00\x00\x00\x83/cf:cq/1697385870854/Put/vlen=4/seqid=0; \x00\x00\x00\x84/cf:cq/1697385870855/Put/vlen=4/seqid=0; \x00\x00\x00\x85/cf:cq/1697385870857/Put/vlen=4/seqid=0; \x00\x00\x00\x86/cf:cq/1697385870859/Put/vlen=4/seqid=0; \x00\x00\x00\x87/cf:cq/1697385870860/Put/vlen=4/seqid=0; \x00\x00\x00\x88/cf:cq/1697385870862/Put/vlen=4/seqid=0; \x00\x00\x00\x89/cf:cq/1697385870864/Put/vlen=4/seqid=0; \x00\x00\x00\x8A/cf:cq/1697385870866/Put/vlen=4/seqid=0; \x00\x00\x00\x8B/cf:cq/1697385870867/Put/vlen=4/seqid=0; \x00\x00\x00\x8C/cf:cq/1697385870869/Put/vlen=4/seqid=0; \x00\x00\x00\x8D/cf:cq/1697385870871/Put/vlen=4/seqid=0; \x00\x00\x00\x8E/cf:cq/1697385870873/Put/vlen=4/seqid=0; \x00\x00\x00\x8F/cf:cq/1697385870875/Put/vlen=4/seqid=0; \x00\x00\x00\x90/cf:cq/1697385870876/Put/vlen=4/seqid=0; \x00\x00\x00\x91/cf:cq/1697385870878/Put/vlen=4/seqid=0; \x00\x00\x00\x92/cf:cq/1697385870881/Put/vlen=4/seqid=0; \x00\x00\x00\x93/cf:cq/1697385870884/Put/vlen=4/seqid=0; \x00\x00\x00\x94/cf:cq/1697385870889/Put/vlen=4/seqid=0; \x00\x00\x00\x95/cf:cq/1697385870892/Put/vlen=4/seqid=0; \x00\x00\x00\x96/cf:cq/1697385870894/Put/vlen=4/seqid=0; \x00\x00\x00\x97/cf:cq/1697385870896/Put/vlen=4/seqid=0; \x00\x00\x00\x98/cf:cq/1697385870898/Put/vlen=4/seqid=0; \x00\x00\x00\x99/cf:cq/1697385870900/Put/vlen=4/seqid=0; \x00\x00\x00\x9A/cf:cq/1697385870902/Put/vlen=4/seqid=0; \x00\x00\x00\x9B/cf:cq/1697385870904/Put/vlen=4/seqid=0; \x00\x00\x00\x9C/cf:cq/1697385870906/Put/vlen=4/seqid=0; \x00\x00\x00\x9D/cf:cq/1697385870908/Put/vlen=4/seqid=0; \x00\x00\x00\x9E/cf:cq/1697385870909/Put/vlen=4/seqid=0; \x00\x00\x00\x9F/cf:cq/1697385870911/Put/vlen=4/seqid=0; \x00\x00\x00\xA0/cf:cq/1697385870915/Put/vlen=4/seqid=0; \x00\x00\x00\xA1/cf:cq/1697385870917/Put/vlen=4/seqid=0; \x00\x00\x00\xA2/cf:cq/1697385870928/Put/vlen=4/seqid=0; \x00\x00\x00\xA3/cf:cq/1697385870930/Put/vlen=4/seqid=0; \x00\x00\x00\xA4/cf:cq/1697385870932/Put/vlen=4/seqid=0; \x00\x00\x00\xA5/cf:cq/1697385870934/Put/vlen=4/seqid=0; \x00\x00\x00\xA6/cf:cq/1697385870936/Put/vlen=4/seqid=0; \x00\x00\x00\xA7/cf:cq/1697385870938/Put/vlen=4/seqid=0; \x00\x00\x00\xA8/cf:cq/1697385870940/Put/vlen=4/seqid=0; \x00\x00\x00\xA9/cf:cq/1697385870942/Put/vlen=4/seqid=0; \x00\x00\x00\xAA/cf:cq/1697385870944/Put/vlen=4/seqid=0; \x00\x00\x00\xAB/cf:cq/1697385870946/Put/vlen=4/seqid=0; \x00\x00\x00\xAC/cf:cq/1697385870948/Put/vlen=4/seqid=0; \x00\x00\x00\xAD/cf:cq/1697385870949/Put/vlen=4/seqid=0; \x00\x00\x00\xAE/cf:cq/1697385870951/Put/vlen=4/seqid=0; \x00\x00\x00\xAF/cf:cq/1697385870954/Put/vlen=4/seqid=0; \x00\x00\x00\xB0/cf:cq/1697385870956/Put/vlen=4/seqid=0; \x00\x00\x00\xB1/cf:cq/1697385870958/Put/vlen=4/seqid=0; \x00\x00\x00\xB2/cf:cq/1697385870960/Put/vlen=4/seqid=0; \x00\x00\x00\xB3/cf:cq/1697385870962/Put/vlen=4/seqid=0; \x00\x00\x00\xB4/cf:cq/1697385870964/Put/vlen=4/seqid=0; \x00\x00\x00\xB5/cf:cq/1697385870966/Put/vlen=4/seqid=0; \x00\x00\x00\xB6/cf:cq/1697385870970/Put/vlen=4/seqid=0; \x00\x00\x00\xB7/cf:cq/1697385870974/Put/vlen=4/seqid=0; \x00\x00\x00\xB8/cf:cq/1697385870976/Put/vlen=4/seqid=0; \x00\x00\x00\xB9/cf:cq/1697385870998/Put/vlen=4/seqid=0; \x00\x00\x00\xBA/cf:cq/1697385871000/Put/vlen=4/seqid=0; \x00\x00\x00\xBB/cf:cq/1697385871002/Put/vlen=4/seqid=0; \x00\x00\x00\xBC/cf:cq/1697385871005/Put/vlen=4/seqid=0; \x00\x00\x00\xBD/cf:cq/1697385871008/Put/vlen=4/seqid=0; \x00\x00\x00\xBE/cf:cq/1697385871010/Put/vlen=4/seqid=0; \x00\x00\x00\xBF/cf:cq/1697385871015/Put/vlen=4/seqid=0; \x00\x00\x00\xC0/cf:cq/1697385871020/Put/vlen=4/seqid=0; \x00\x00\x00\xC1/cf:cq/1697385871023/Put/vlen=4/seqid=0; \x00\x00\x00\xC2/cf:cq/1697385871028/Put/vlen=4/seqid=0; \x00\x00\x00\xC3/cf:cq/1697385871031/Put/vlen=4/seqid=0; \x00\x00\x00\xC4/cf:cq/1697385871034/Put/vlen=4/seqid=0; \x00\x00\x00\xC5/cf:cq/1697385871038/Put/vlen=4/seqid=0; \x00\x00\x00\xC6/cf:cq/1697385871041/Put/vlen=4/seqid=0; \x00\x00\x00\xC7/cf:cq/1697385871044/Put/vlen=4/seqid=0; \x00\x00\x00\xC8/cf:cq/1697385871048/Put/vlen=4/seqid=0; \x00\x00\x00\xC9/cf:cq/1697385871051/Put/vlen=4/seqid=0; \x00\x00\x00\xCA/cf:cq/1697385871053/Put/vlen=4/seqid=0; \x00\x00\x00\xCB/cf:cq/1697385871056/Put/vlen=4/seqid=0; \x00\x00\x00\xCC/cf:cq/1697385871058/Put/vlen=4/seqid=0; \x00\x00\x00\xCD/cf:cq/1697385871060/Put/vlen=4/seqid=0; \x00\x00\x00\xCE/cf:cq/1697385871063/Put/vlen=4/seqid=0; \x00\x00\x00\xCF/cf:cq/1697385871065/Put/vlen=4/seqid=0; \x00\x00\x00\xD0/cf:cq/1697385871068/Put/vlen=4/seqid=0; \x00\x00\x00\xD1/cf:cq/1697385871070/Put/vlen=4/seqid=0; \x00\x00\x00\xD2/cf:cq/1697385871072/Put/vlen=4/seqid=0; \x00\x00\x00\xD3/cf:cq/1697385871075/Put/vlen=4/seqid=0; \x00\x00\x00\xD4/cf:cq/1697385871077/Put/vlen=4/seqid=0; \x00\x00\x00\xD5/cf:cq/1697385871086/Put/vlen=4/seqid=0; \x00\x00\x00\xD6/cf:cq/1697385871091/Put/vlen=4/seqid=0; \x00\x00\x00\xD7/cf:cq/1697385871094/Put/vlen=4/seqid=0; \x00\x00\x00\xD8/cf:cq/1697385871097/Put/vlen=4/seqid=0; \x00\x00\x00\xD9/cf:cq/1697385871100/Put/vlen=4/seqid=0; \x00\x00\x00\xDA/cf:cq/1697385871102/Put/vlen=4/seqid=0; \x00\x00\x00\xDB/cf:cq/1697385871105/Put/vlen=4/seqid=0; \x00\x00\x00\xDC/cf:cq/1697385871107/Put/vlen=4/seqid=0; \x00\x00\x00\xDD/cf:cq/1697385871109/Put/vlen=4/seqid=0; \x00\x00\x00\xDE/cf:cq/1697385871111/Put/vlen=4/seqid=0; \x00\x00\x00\xDF/cf:cq/1697385871113/Put/vlen=4/seqid=0; \x00\x00\x00\xE0/cf:cq/1697385871115/Put/vlen=4/seqid=0; \x00\x00\x00\xE1/cf:cq/1697385871121/Put/vlen=4/seqid=0; \x00\x00\x00\xE2/cf:cq/1697385871124/Put/vlen=4/seqid=0; \x00\x00\x00\xE3/cf:cq/1697385871126/Put/vlen=4/seqid=0; \x00\x00\x00\xE4/cf:cq/1697385871129/Put/vlen=4/seqid=0; \x00\x00\x00\xE5/cf:cq/1697385871131/Put/vlen=4/seqid=0; \x00\x00\x00\xE6/cf:cq/1697385871133/Put/vlen=4/seqid=0; \x00\x00\x00\xE7/cf:cq/1697385871136/Put/vlen=4/seqid=0; \x00\x00\x00\xE8/cf:cq/1697385871138/Put/vlen=4/seqid=0; \x00\x00\x00\xE9/cf:cq/1697385871140/Put/vlen=4/seqid=0; \x00\x00\x00\xEA/cf:cq/1697385871142/Put/vlen=4/seqid=0; \x00\x00\x00\xEB/cf:cq/1697385871144/Put/vlen=4/seqid=0; \x00\x00\x00\xEC/cf:cq/1697385871146/Put/vlen=4/seqid=0; \x00\x00\x00\xED/cf:cq/1697385871148/Put/vlen=4/seqid=0; \x00\x00\x00\xEE/cf:cq/1697385871150/Put/vlen=4/seqid=0; \x00\x00\x00\xEF/cf:cq/1697385871152/Put/vlen=4/seqid=0; \x00\x00\x00\xF0/cf:cq/1697385871154/Put/vlen=4/seqid=0; \x00\x00\x00\xF1/cf:cq/1697385871155/Put/vlen=4/seqid=0; \x00\x00\x00\xF2/cf:cq/1697385871158/Put/vlen=4/seqid=0; \x00\x00\x00\xF3/cf:cq/1697385871160/Put/vlen=4/seqid=0; \x00\x00\x00\xF4/cf:cq/1697385871162/Put/vlen=4/seqid=0; \x00\x00\x00\xF5/cf:cq/1697385871164/Put/vlen=4/seqid=0; \x00\x00\x00\xF6/cf:cq/1697385871166/Put/vlen=4/seqid=0; \x00\x00\x00\xF7/cf:cq/1697385871167/Put/vlen=4/seqid=0; \x00\x00\x00\xF8/cf:cq/1697385871169/Put/vlen=4/seqid=0; \x00\x00\x00\xF9/cf:cq/1697385871170/Put/vlen=4/seqid=0; \x00\x00\x00\xFA/cf:cq/1697385871172/Put/vlen=4/seqid=0; \x00\x00\x00\xFB/cf:cq/1697385871174/Put/vlen=4/seqid=0; \x00\x00\x00\xFC/cf:cq/1697385871176/Put/vlen=4/seqid=0; \x00\x00\x00\xFD/cf:cq/1697385871178/Put/vlen=4/seqid=0; \x00\x00\x00\xFE/cf:cq/1697385871179/Put/vlen=4/seqid=0; \x00\x00\x00\xFF/cf:cq/1697385871182/Put/vlen=4/seqid=0; \x00\x00\x01\x00/cf:cq/1697385871187/Put/vlen=4/seqid=0; \x00\x00\x01\x01/cf:cq/1697385871189/Put/vlen=4/seqid=0; \x00\x00\x01\x02/cf:cq/1697385871193/Put/vlen=4/seqid=0; \x00\x00\x01\x03/cf:cq/1697385871196/Put/vlen=4/seqid=0; \x00\x00\x01\x04/cf:cq/1697385871198/Put/vlen=4/seqid=0; \x00\x00\x01\x05/cf:cq/1697385871199/Put/vlen=4/seqid=0; \x00\x00\x01\x06/cf:cq/1697385871201/Put/vlen=4/seqid=0; \x00\x00\x01\x07/cf:cq/1697385871203/Put/vlen=4/seqid=0; \x00\x00\x01\x08/cf:cq/1697385871204/Put/vlen=4/seqid=0; \x00\x00\x01\x09/cf:cq/1697385871207/Put/vlen=4/seqid=0; \x00\x00\x01\x0A/cf:cq/1697385871209/Put/vlen=4/seqid=0; \x00\x00\x01\x0B/cf:cq/1697385871211/Put/vlen=4/seqid=0; \x00\x00\x01\x0C/cf:cq/1697385871212/Put/vlen=4/seqid=0; \x00\x00\x01\x0D/cf:cq/1697385871214/Put/vlen=4/seqid=0; \x00\x00\x01\x0E/cf:cq/1697385871216/Put/vlen=4/seqid=0; \x00\x00\x01\x0F/cf:cq/1697385871218/Put/vlen=4/seqid=0; \x00\x00\x01\x10/cf:cq/1697385871221/Put/vlen=4/seqid=0; \x00\x00\x01\x11/cf:cq/1697385871223/Put/vlen=4/seqid=0; \x00\x00\x01\x12/cf:cq/1697385871229/Put/vlen=4/seqid=0; \x00\x00\x01\x13/cf:cq/1697385871231/Put/vlen=4/seqid=0; \x00\x00\x01\x14/cf:cq/1697385871233/Put/vlen=4/seqid=0; \x00\x00\x01\x15/cf:cq/1697385871235/Put/vlen=4/seqid=0; \x00\x00\x01\x16/cf:cq/1697385871237/Put/vlen=4/seqid=0; \x00\x00\x01\x17/cf:cq/1697385871239/Put/vlen=4/seqid=0; \x00\x00\x01\x18/cf:cq/1697385871241/Put/vlen=4/seqid=0; \x00\x00\x01\x19/cf:cq/1697385871243/Put/vlen=4/seqid=0; \x00\x00\x01\x1A/cf:cq/1697385871244/Put/vlen=4/seqid=0; \x00\x00\x01\x1B/cf:cq/1697385871246/Put/vlen=4/seqid=0; \x00\x00\x01\x1C/cf:cq/1697385871248/Put/vlen=4/seqid=0; \x00\x00\x01\x1D/cf:cq/1697385871250/Put/vlen=4/seqid=0; \x00\x00\x01\x1E/cf:cq/1697385871251/Put/vlen=4/seqid=0; \x00\x00\x01\x1F/cf:cq/1697385871253/Put/vlen=4/seqid=0; \x00\x00\x01 /cf:cq/1697385871257/Put/vlen=4/seqid=0; \x00\x00\x01!/cf:cq/1697385871260/Put/vlen=4/seqid=0; \x00\x00\x01"/cf:cq/1697385871261/Put/vlen=4/seqid=0; \x00\x00\x01#/cf:cq/1697385871263/Put/vlen=4/seqid=0; \x00\x00\x01$/cf:cq/1697385871264/Put/vlen=4/seqid=0; \x00\x00\x01%/cf:cq/1697385871266/Put/vlen=4/seqid=0; \x00\x00\x01&/cf:cq/1697385871268/Put/vlen=4/seqid=0; \x00\x00\x01'/cf:cq/1697385871270/Put/vlen=4/seqid=0; \x00\x00\x01(/cf:cq/1697385871271/Put/vlen=4/seqid=0; \x00\x00\x01)/cf:cq/1697385871274/Put/vlen=4/seqid=0; \x00\x00\x01*/cf:cq/1697385871278/Put/vlen=4/seqid=0; \x00\x00\x01+/cf:cq/1697385871279/Put/vlen=4/seqid=0; \x00\x00\x01,/cf:cq/1697385871282/Put/vlen=4/seqid=0; \x00\x00\x01-/cf:cq/1697385871285/Put/vlen=4/seqid=0; \x00\x00\x01./cf:cq/1697385871289/Put/vlen=4/seqid=0; \x00\x00\x01//cf:cq/1697385871291/Put/vlen=4/seqid=0; \x00\x00\x010/cf:cq/1697385871293/Put/vlen=4/seqid=0; \x00\x00\x011/cf:cq/1697385871297/Put/vlen=4/seqid=0; \x00\x00\x012/cf:cq/1697385871307/Put/vlen=4/seqid=0; \x00\x00\x013/cf:cq/1697385871310/Put/vlen=4/seqid=0; \x00\x00\x014/cf:cq/1697385871312/Put/vlen=4/seqid=0; \x00\x00\x015/cf:cq/1697385871314/Put/vlen=4/seqid=0; \x00\x00\x016/cf:cq/1697385871316/Put/vlen=4/seqid=0; \x00\x00\x017/cf:cq/1697385871318/Put/vlen=4/seqid=0; \x00\x00\x018/cf:cq/1697385871320/Put/vlen=4/seqid=0; \x00\x00\x019/cf:cq/1697385871322/Put/vlen=4/seqid=0; \x00\x00\x01:/cf:cq/1697385871325/Put/vlen=4/seqid=0; \x00\x00\x01;/cf:cq/1697385871327/Put/vlen=4/seqid=0; \x00\x00\x01/cf:cq/1697385871333/Put/vlen=4/seqid=0; \x00\x00\x01?/cf:cq/1697385871338/Put/vlen=4/seqid=0; \x00\x00\x01@/cf:cq/1697385871340/Put/vlen=4/seqid=0; \x00\x00\x01A/cf:cq/1697385871342/Put/vlen=4/seqid=0; \x00\x00\x01B/cf:cq/1697385871345/Put/vlen=4/seqid=0; \x00\x00\x01C/cf:cq/1697385871346/Put/vlen=4/seqid=0; \x00\x00\x01D/cf:cq/1697385871349/Put/vlen=4/seqid=0; \x00\x00\x01E/cf:cq/1697385871350/Put/vlen=4/seqid=0; \x00\x00\x01F/cf:cq/1697385871352/Put/vlen=4/seqid=0; \x00\x00\x01G/cf:cq/1697385871354/Put/vlen=4/seqid=0; \x00\x00\x01H/cf:cq/1697385871356/Put/vlen=4/seqid=0; \x00\x00\x01I/cf:cq/1697385871358/Put/vlen=4/seqid=0; \x00\x00\x01J/cf:cq/1697385871360/Put/vlen=4/seqid=0; \x00\x00\x01K/cf:cq/1697385871362/Put/vlen=4/seqid=0; \x00\x00\x01L/cf:cq/1697385871367/Put/vlen=4/seqid=0; \x00\x00\x01M/cf:cq/1697385871372/Put/vlen=4/seqid=0; \x00\x00\x01N/cf:cq/1697385871374/Put/vlen=4/seqid=0; \x00\x00\x01O/cf:cq/1697385871376/Put/vlen=4/seqid=0; \x00\x00\x01P/cf:cq/1697385871377/Put/vlen=4/seqid=0; \x00\x00\x01Q/cf:cq/1697385871380/Put/vlen=4/seqid=0; \x00\x00\x01R/cf:cq/1697385871381/Put/vlen=4/seqid=0; \x00\x00\x01S/cf:cq/1697385871383/Put/vlen=4/seqid=0; \x00\x00\x01T/cf:cq/1697385871385/Put/vlen=4/seqid=0; \x00\x00\x01U/cf:cq/1697385871387/Put/vlen=4/seqid=0; \x00\x00\x01V/cf:cq/1697385871389/Put/vlen=4/seqid=0; \x00\x00\x01W/cf:cq/1697385871390/Put/vlen=4/seqid=0; \x00\x00\x01X/cf:cq/1697385871394/Put/vlen=4/seqid=0; \x00\x00\x01Y/cf:cq/1697385871396/Put/vlen=4/seqid=0; \x00\x00\x01Z/cf:cq/1697385871400/Put/vlen=4/seqid=0; \x00\x00\x01[/cf:cq/1697385871402/Put/vlen=4/seqid=0; \x00\x00\x01\x5C/cf:cq/1697385871404/Put/vlen=4/seqid=0; \x00\x00\x01]/cf:cq/1697385871405/Put/vlen=4/seqid=0; \x00\x00\x01^/cf:cq/1697385871407/Put/vlen=4/seqid=0; \x00\x00\x01_/cf:cq/1697385871409/Put/vlen=4/seqid=0; \x00\x00\x01`/cf:cq/1697385871411/Put/vlen=4/seqid=0; \x00\x00\x01a/cf:cq/1697385871412/Put/vlen=4/seqid=0; \x00\x00\x01b/cf:cq/1697385871414/Put/vlen=4/seqid=0; \x00\x00\x01c/cf:cq/1697385871416/Put/vlen=4/seqid=0; \x00\x00\x01d/cf:cq/1697385871417/Put/vlen=4/seqid=0; \x00\x00\x01e/cf:cq/1697385871419/Put/vlen=4/seqid=0; \x00\x00\x01f/cf:cq/1697385871421/Put/vlen=4/seqid=0; \x00\x00\x01g/cf:cq/1697385871423/Put/vlen=4/seqid=0; \x00\x00\x01h/cf:cq/1697385871426/Put/vlen=4/seqid=0; \x00\x00\x01i/cf:cq/1697385871428/Put/vlen=4/seqid=0; \x00\x00\x01j/cf:cq/1697385871430/Put/vlen=4/seqid=0; \x00\x00\x01k/cf:cq/1697385871431/Put/vlen=4/seqid=0; \x00\x00\x01l/cf:cq/1697385871433/Put/vlen=4/seqid=0; \x00\x00\x01m/cf:cq/1697385871434/Put/vlen=4/seqid=0; \x00\x00\x01n/cf:cq/1697385871436/Put/vlen=4/seqid=0; \x00\x00\x01o/cf:cq/1697385871438/Put/vlen=4/seqid=0; \x00\x00\x01p/cf:cq/1697385871439/Put/vlen=4/seqid=0; \x00\x00\x01q/cf:cq/1697385871441/Put/vlen=4/seqid=0; \x00\x00\x01r/cf:cq/1697385871442/Put/vlen=4/seqid=0; \x00\x00\x01s/cf:cq/1697385871444/Put/vlen=4/seqid=0; \x00\x00\x01t/cf:cq/1697385871446/Put/vlen=4/seqid=0; \x00\x00\x01u/cf:cq/1697385871448/Put/vlen=4/seqid=0; \x00\x00\x01v/cf:cq/1697385871450/Put/vlen=4/seqid=0; \x00\x00\x01w/cf:cq/1697385871452/Put/vlen=4/seqid=0; \x00\x00\x01x/cf:cq/1697385871453/Put/vlen=4/seqid=0; \x00\x00\x01y/cf:cq/1697385871455/Put/vlen=4/seqid=0; \x00\x00\x01z/cf:cq/1697385871457/Put/vlen=4/seqid=0; \x00\x00\x01{/cf:cq/1697385871459/Put/vlen=4/seqid=0; \x00\x00\x01|/cf:cq/1697385871461/Put/vlen=4/seqid=0; \x00\x00\x01}/cf:cq/1697385871463/Put/vlen=4/seqid=0; \x00\x00\x01~/cf:cq/1697385871465/Put/vlen=4/seqid=0; \x00\x00\x01\x7F/cf:cq/1697385871466/Put/vlen=4/seqid=0; \x00\x00\x01\x80/cf:cq/1697385871468/Put/vlen=4/seqid=0; \x00\x00\x01\x81/cf:cq/1697385871470/Put/vlen=4/seqid=0; \x00\x00\x01\x82/cf:cq/1697385871471/Put/vlen=4/seqid=0; \x00\x00\x01\x83/cf:cq/1697385871473/Put/vlen=4/seqid=0; \x00\x00\x01\x84/cf:cq/1697385871475/Put/vlen=4/seqid=0; \x00\x00\x01\x85/cf:cq/1697385871477/Put/vlen=4/seqid=0; \x00\x00\x01\x86/cf:cq/1697385871479/Put/vlen=4/seqid=0; \x00\x00\x01\x87/cf:cq/1697385871482/Put/vlen=4/seqid=0; \x00\x00\x01\x88/cf:cq/1697385871484/Put/vlen=4/seqid=0; \x00\x00\x01\x89/cf:cq/1697385871486/Put/vlen=4/seqid=0; \x00\x00\x01\x8A/cf:cq/1697385871487/Put/vlen=4/seqid=0; \x00\x00\x01\x8B/cf:cq/1697385871489/Put/vlen=4/seqid=0; \x00\x00\x01\x8C/cf:cq/1697385871491/Put/vlen=4/seqid=0; \x00\x00\x01\x8D/cf:cq/1697385871493/Put/vlen=4/seqid=0; \x00\x00\x01\x8E/cf:cq/1697385871496/Put/vlen=4/seqid=0; \x00\x00\x01\x8F/cf:cq/1697385871498/Put/vlen=4/seqid=0; \x00\x00\x01\x90/cf:cq/1697385871505/Put/vlen=4/seqid=0; \x00\x00\x01\x91/cf:cq/1697385871508/Put/vlen=4/seqid=0; \x00\x00\x01\x92/cf:cq/1697385871510/Put/vlen=4/seqid=0; \x00\x00\x01\x93/cf:cq/1697385871517/Put/vlen=4/seqid=0; \x00\x00\x01\x94/cf:cq/1697385871519/Put/vlen=4/seqid=0; \x00\x00\x01\x95/cf:cq/1697385871521/Put/vlen=4/seqid=0; \x00\x00\x01\x96/cf:cq/1697385871523/Put/vlen=4/seqid=0; \x00\x00\x01\x97/cf:cq/1697385871525/Put/vlen=4/seqid=0; \x00\x00\x01\x98/cf:cq/1697385871530/Put/vlen=4/seqid=0; \x00\x00\x01\x99/cf:cq/1697385871532/Put/vlen=4/seqid=0; \x00\x00\x01\x9A/cf:cq/1697385871534/Put/vlen=4/seqid=0; \x00\x00\x01\x9B/cf:cq/1697385871536/Put/vlen=4/seqid=0; \x00\x00\x01\x9C/cf:cq/1697385871538/Put/vlen=4/seqid=0; \x00\x00\x01\x9D/cf:cq/1697385871540/Put/vlen=4/seqid=0; \x00\x00\x01\x9E/cf:cq/1697385871542/Put/vlen=4/seqid=0; \x00\x00\x01\x9F/cf:cq/1697385871544/Put/vlen=4/seqid=0; \x00\x00\x01\xA0/cf:cq/1697385871546/Put/vlen=4/seqid=0; \x00\x00\x01\xA1/cf:cq/1697385871549/Put/vlen=4/seqid=0; \x00\x00\x01\xA2/cf:cq/1697385871551/Put/vlen=4/seqid=0; \x00\x00\x01\xA3/cf:cq/1697385871553/Put/vlen=4/seqid=0; \x00\x00\x01\xA4/cf:cq/1697385871555/Put/vlen=4/seqid=0; \x00\x00\x01\xA5/cf:cq/1697385871557/Put/vlen=4/seqid=0; \x00\x00\x01\xA6/cf:cq/1697385871559/Put/vlen=4/seqid=0; \x00\x00\x01\xA7/cf:cq/1697385871561/Put/vlen=4/seqid=0; \x00\x00\x01\xA8/cf:cq/1697385871565/Put/vlen=4/seqid=0; \x00\x00\x01\xA9/cf:cq/1697385871566/Put/vlen=4/seqid=0; \x00\x00\x01\xAA/cf:cq/1697385871568/Put/vlen=4/seqid=0; \x00\x00\x01\xAB/cf:cq/1697385871570/Put/vlen=4/seqid=0; \x00\x00\x01\xAC/cf:cq/1697385871572/Put/vlen=4/seqid=0; \x00\x00\x01\xAD/cf:cq/1697385871574/Put/vlen=4/seqid=0; \x00\x00\x01\xAE/cf:cq/1697385871576/Put/vlen=4/seqid=0; \x00\x00\x01\xAF/cf:cq/1697385871581/Put/vlen=4/seqid=0; \x00\x00\x01\xB0/cf:cq/1697385871583/Put/vlen=4/seqid=0; \x00\x00\x01\xB1/cf:cq/1697385871585/Put/vlen=4/seqid=0; \x00\x00\x01\xB2/cf:cq/1697385871587/Put/vlen=4/seqid=0; \x00\x00\x01\xB3/cf:cq/1697385871589/Put/vlen=4/seqid=0; \x00\x00\x01\xB4/cf:cq/1697385871591/Put/vlen=4/seqid=0; \x00\x00\x01\xB5/cf:cq/1697385871593/Put/vlen=4/seqid=0; \x00\x00\x01\xB6/cf:cq/1697385871594/Put/vlen=4/seqid=0; \x00\x00\x01\xB7/cf:cq/1697385871597/Put/vlen=4/seqid=0; \x00\x00\x01\xB8/cf:cq/1697385871599/Put/vlen=4/seqid=0; \x00\x00\x01\xB9/cf:cq/1697385871600/Put/vlen=4/seqid=0; \x00\x00\x01\xBA/cf:cq/1697385871602/Put/vlen=4/seqid=0; \x00\x00\x01\xBB/cf:cq/1697385871604/Put/vlen=4/seqid=0; \x00\x00\x01\xBC/cf:cq/1697385871606/Put/vlen=4/seqid=0; \x00\x00\x01\xBD/cf:cq/1697385871618/Put/vlen=4/seqid=0; \x00\x00\x01\xBE/cf:cq/1697385871620/Put/vlen=4/seqid=0; \x00\x00\x01\xBF/cf:cq/1697385871622/Put/vlen=4/seqid=0; \x00\x00\x01\xC0/cf:cq/1697385871624/Put/vlen=4/seqid=0; \x00\x00\x01\xC1/cf:cq/1697385871626/Put/vlen=4/seqid=0; \x00\x00\x01\xC2/cf:cq/1697385871627/Put/vlen=4/seqid=0; \x00\x00\x01\xC3/cf:cq/1697385871631/Put/vlen=4/seqid=0; \x00\x00\x01\xC4/cf:cq/1697385871633/Put/vlen=4/seqid=0; \x00\x00\x01\xC5/cf:cq/1697385871634/Put/vlen=4/seqid=0; \x00\x00\x01\xC6/cf:cq/1697385871636/Put/vlen=4/seqid=0; \x00\x00\x01\xC7/cf:cq/1697385871638/Put/vlen=4/seqid=0; \x00\x00\x01\xC8/cf:cq/1697385871640/Put/vlen=4/seqid=0; \x00\x00\x01\xC9/cf:cq/1697385871642/Put/vlen=4/seqid=0; \x00\x00\x01\xCA/cf:cq/1697385871644/Put/vlen=4/seqid=0; \x00\x00\x01\xCB/cf:cq/1697385871645/Put/vlen=4/seqid=0; \x00\x00\x01\xCC/cf:cq/1697385871646/Put/vlen=4/seqid=0; \x00\x00\x01\xCD/cf:cq/1697385871648/Put/vlen=4/seqid=0; \x00\x00\x01\xCE/cf:cq/1697385871650/Put/vlen=4/seqid=0; \x00\x00\x01\xCF/cf:cq/1697385871651/Put/vlen=4/seqid=0; \x00\x00\x01\xD0/cf:cq/1697385871653/Put/vlen=4/seqid=0; \x00\x00\x01\xD1/cf:cq/1697385871654/Put/vlen=4/seqid=0; \x00\x00\x01\xD2/cf:cq/1697385871657/Put/vlen=4/seqid=0; \x00\x00\x01\xD3/cf:cq/1697385871659/Put/vlen=4/seqid=0; \x00\x00\x01\xD4/cf:cq/1697385871660/Put/vlen=4/seqid=0; \x00\x00\x01\xD5/cf:cq/1697385871662/Put/vlen=4/seqid=0; \x00\x00\x01\xD6/cf:cq/1697385871664/Put/vlen=4/seqid=0; \x00\x00\x01\xD7/cf:cq/1697385871666/Put/vlen=4/seqid=0; \x00\x00\x01\xD8/cf:cq/1697385871668/Put/vlen=4/seqid=0; \x00\x00\x01\xD9/cf:cq/1697385871669/Put/vlen=4/seqid=0; \x00\x00\x01\xDA/cf:cq/1697385871671/Put/vlen=4/seqid=0; \x00\x00\x01\xDB/cf:cq/1697385871672/Put/vlen=4/seqid=0; \x00\x00\x01\xDC/cf:cq/1697385871674/Put/vlen=4/seqid=0; \x00\x00\x01\xDD/cf:cq/1697385871676/Put/vlen=4/seqid=0; \x00\x00\x01\xDE/cf:cq/1697385871677/Put/vlen=4/seqid=0; \x00\x00\x01\xDF/cf:cq/1697385871679/Put/vlen=4/seqid=0; \x00\x00\x01\xE0/cf:cq/1697385871681/Put/vlen=4/seqid=0; \x00\x00\x01\xE1/cf:cq/1697385871682/Put/vlen=4/seqid=0; \x00\x00\x01\xE2/cf:cq/1697385871685/Put/vlen=4/seqid=0; \x00\x00\x01\xE3/cf:cq/1697385871687/Put/vlen=4/seqid=0; \x00\x00\x01\xE4/cf:cq/1697385871688/Put/vlen=4/seqid=0; \x00\x00\x01\xE5/cf:cq/1697385871689/Put/vlen=4/seqid=0; \x00\x00\x01\xE6/cf:cq/1697385871691/Put/vlen=4/seqid=0; \x00\x00\x01\xE7/cf:cq/1697385871693/Put/vlen=4/seqid=0; \x00\x00\x01\xE8/cf:cq/1697385871694/Put/vlen=4/seqid=0; \x00\x00\x01\xE9/cf:cq/1697385871696/Put/vlen=4/seqid=0; \x00\x00\x01\xEA/cf:cq/1697385871697/Put/vlen=4/seqid=0; \x00\x00\x01\xEB/cf:cq/1697385871699/Put/vlen=4/seqid=0; \x00\x00\x01\xEC/cf:cq/1697385871701/Put/vlen=4/seqid=0; \x00\x00\x01\xED/cf:cq/1697385871702/Put/vlen=4/seqid=0; \x00\x00\x01\xEE/cf:cq/1697385871704/Put/vlen=4/seqid=0; \x00\x00\x01\xEF/cf:cq/1697385871706/Put/vlen=4/seqid=0; \x00\x00\x01\xF0/cf:cq/1697385871708/Put/vlen=4/seqid=0; \x00\x00\x01\xF1/cf:cq/1697385871710/Put/vlen=4/seqid=0; \x00\x00\x01\xF2/cf:cq/1697385871711/Put/vlen=4/seqid=0; \x00\x00\x01\xF3/cf:cq/1697385871712/Put/vlen=4/seqid=0; \x00\x00\x01\xF4/cf:cq/1697385871714/Put/vlen=4/seqid=0; \x00\x00\x01\xF5/cf:cq/1697385871716/Put/vlen=4/seqid=0; \x00\x00\x01\xF6/cf:cq/1697385871718/Put/vlen=4/seqid=0; \x00\x00\x01\xF7/cf:cq/1697385871719/Put/vlen=4/seqid=0; \x00\x00\x01\xF8/cf:cq/1697385871721/Put/vlen=4/seqid=0; \x00\x00\x01\xF9/cf:cq/1697385871723/Put/vlen=4/seqid=0; \x00\x00\x01\xFA/cf:cq/1697385871725/Put/vlen=4/seqid=0; \x00\x00\x01\xFB/cf:cq/1697385871726/Put/vlen=4/seqid=0; \x00\x00\x01\xFC/cf:cq/1697385871728/Put/vlen=4/seqid=0; \x00\x00\x01\xFD/cf:cq/1697385871731/Put/vlen=4/seqid=0; \x00\x00\x01\xFE/cf:cq/1697385871734/Put/vlen=4/seqid=0; \x00\x00\x01\xFF/cf:cq/1697385871736/Put/vlen=4/seqid=0; \x00\x00\x02\x00/cf:cq/1697385871737/Put/vlen=4/seqid=0; \x00\x00\x02\x01/cf:cq/1697385871739/Put/vlen=4/seqid=0; \x00\x00\x02\x02/cf:cq/1697385871740/Put/vlen=4/seqid=0; \x00\x00\x02\x03/cf:cq/1697385871742/Put/vlen=4/seqid=0; \x00\x00\x02\x04/cf:cq/1697385871743/Put/vlen=4/seqid=0; \x00\x00\x02\x05/cf:cq/1697385871745/Put/vlen=4/seqid=0; \x00\x00\x02\x06/cf:cq/1697385871746/Put/vlen=4/seqid=0; \x00\x00\x02\x07/cf:cq/1697385871748/Put/vlen=4/seqid=0; \x00\x00\x02\x08/cf:cq/1697385871749/Put/vlen=4/seqid=0; \x00\x00\x02\x09/cf:cq/1697385871751/Put/vlen=4/seqid=0; \x00\x00\x02\x0A/cf:cq/1697385871753/Put/vlen=4/seqid=0; \x00\x00\x02\x0B/cf:cq/1697385871754/Put/vlen=4/seqid=0; \x00\x00\x02\x0C/cf:cq/1697385871756/Put/vlen=4/seqid=0; \x00\x00\x02\x0D/cf:cq/1697385871757/Put/vlen=4/seqid=0; \x00\x00\x02\x0E/cf:cq/1697385871759/Put/vlen=4/seqid=0; \x00\x00\x02\x0F/cf:cq/1697385871761/Put/vlen=4/seqid=0; \x00\x00\x02\x10/cf:cq/1697385871763/Put/vlen=4/seqid=0; \x00\x00\x02\x11/cf:cq/1697385871765/Put/vlen=4/seqid=0; \x00\x00\x02\x12/cf:cq/1697385871767/Put/vlen=4/seqid=0; \x00\x00\x02\x13/cf:cq/1697385871769/Put/vlen=4/seqid=0; \x00\x00\x02\x14/cf:cq/1697385871770/Put/vlen=4/seqid=0; \x00\x00\x02\x15/cf:cq/1697385871772/Put/vlen=4/seqid=0; \x00\x00\x02\x16/cf:cq/1697385871773/Put/vlen=4/seqid=0; \x00\x00\x02\x17/cf:cq/1697385871775/Put/vlen=4/seqid=0; \x00\x00\x02\x18/cf:cq/1697385871777/Put/vlen=4/seqid=0; \x00\x00\x02\x19/cf:cq/1697385871778/Put/vlen=4/seqid=0; \x00\x00\x02\x1A/cf:cq/1697385871780/Put/vlen=4/seqid=0; \x00\x00\x02\x1B/cf:cq/1697385871781/Put/vlen=4/seqid=0; \x00\x00\x02\x1C/cf:cq/1697385871783/Put/vlen=4/seqid=0; \x00\x00\x02\x1D/cf:cq/1697385871785/Put/vlen=4/seqid=0; \x00\x00\x02\x1E/cf:cq/1697385871786/Put/vlen=4/seqid=0; \x00\x00\x02\x1F/cf:cq/1697385871788/Put/vlen=4/seqid=0; \x00\x00\x02 /cf:cq/1697385871789/Put/vlen=4/seqid=0; \x00\x00\x02!/cf:cq/1697385871791/Put/vlen=4/seqid=0; \x00\x00\x02"/cf:cq/1697385871793/Put/vlen=4/seqid=0; \x00\x00\x02#/cf:cq/1697385871794/Put/vlen=4/seqid=0; \x00\x00\x02$/cf:cq/1697385871796/Put/vlen=4/seqid=0; \x00\x00\x02%/cf:cq/1697385871797/Put/vlen=4/seqid=0; \x00\x00\x02&/cf:cq/1697385871799/Put/vlen=4/seqid=0; \x00\x00\x02'/cf:cq/1697385871801/Put/vlen=4/seqid=0; \x00\x00\x02(/cf:cq/1697385871803/Put/vlen=4/seqid=0; \x00\x00\x02)/cf:cq/1697385871804/Put/vlen=4/seqid=0; \x00\x00\x02*/cf:cq/1697385871806/Put/vlen=4/seqid=0; \x00\x00\x02+/cf:cq/1697385871817/Put/vlen=4/seqid=0; \x00\x00\x02,/cf:cq/1697385871819/Put/vlen=4/seqid=0; \x00\x00\x02-/cf:cq/1697385871822/Put/vlen=4/seqid=0; \x00\x00\x02./cf:cq/1697385871823/Put/vlen=4/seqid=0; \x00\x00\x02//cf:cq/1697385871824/Put/vlen=4/seqid=0; \x00\x00\x020/cf:cq/1697385871826/Put/vlen=4/seqid=0; \x00\x00\x021/cf:cq/1697385871827/Put/vlen=4/seqid=0; \x00\x00\x022/cf:cq/1697385871829/Put/vlen=4/seqid=0; \x00\x00\x023/cf:cq/1697385871837/Put/vlen=4/seqid=0; \x00\x00\x024/cf:cq/1697385871839/Put/vlen=4/seqid=0; \x00\x00\x025/cf:cq/1697385871840/Put/vlen=4/seqid=0; \x00\x00\x026/cf:cq/1697385871841/Put/vlen=4/seqid=0; \x00\x00\x027/cf:cq/1697385871843/Put/vlen=4/seqid=0; \x00\x00\x028/cf:cq/1697385871844/Put/vlen=4/seqid=0; \x00\x00\x029/cf:cq/1697385871845/Put/vlen=4/seqid=0; \x00\x00\x02:/cf:cq/1697385871847/Put/vlen=4/seqid=0; \x00\x00\x02;/cf:cq/1697385871848/Put/vlen=4/seqid=0; \x00\x00\x02/cf:cq/1697385871852/Put/vlen=4/seqid=0; \x00\x00\x02?/cf:cq/1697385871854/Put/vlen=4/seqid=0; \x00\x00\x02@/cf:cq/1697385871855/Put/vlen=4/seqid=0; \x00\x00\x02A/cf:cq/1697385871856/Put/vlen=4/seqid=0; \x00\x00\x02B/cf:cq/1697385871858/Put/vlen=4/seqid=0; \x00\x00\x02C/cf:cq/1697385871859/Put/vlen=4/seqid=0; \x00\x00\x02D/cf:cq/1697385871861/Put/vlen=4/seqid=0; \x00\x00\x02E/cf:cq/1697385871862/Put/vlen=4/seqid=0; \x00\x00\x02F/cf:cq/1697385871863/Put/vlen=4/seqid=0; \x00\x00\x02G/cf:cq/1697385871864/Put/vlen=4/seqid=0; \x00\x00\x02H/cf:cq/1697385871866/Put/vlen=4/seqid=0; \x00\x00\x02I/cf:cq/1697385871867/Put/vlen=4/seqid=0; \x00\x00\x02J/cf:cq/1697385871869/Put/vlen=4/seqid=0; \x00\x00\x02K/cf:cq/1697385871870/Put/vlen=4/seqid=0; \x00\x00\x02L/cf:cq/1697385871872/Put/vlen=4/seqid=0; \x00\x00\x02M/cf:cq/1697385871873/Put/vlen=4/seqid=0; \x00\x00\x02N/cf:cq/1697385871874/Put/vlen=4/seqid=0; \x00\x00\x02O/cf:cq/1697385871877/Put/vlen=4/seqid=0; \x00\x00\x02P/cf:cq/1697385871878/Put/vlen=4/seqid=0; \x00\x00\x02Q/cf:cq/1697385871880/Put/vlen=4/seqid=0; \x00\x00\x02R/cf:cq/1697385871881/Put/vlen=4/seqid=0; \x00\x00\x02S/cf:cq/1697385871882/Put/vlen=4/seqid=0; \x00\x00\x02T/cf:cq/1697385871884/Put/vlen=4/seqid=0; \x00\x00\x02U/cf:cq/1697385871885/Put/vlen=4/seqid=0; \x00\x00\x02V/cf:cq/1697385871886/Put/vlen=4/seqid=0; \x00\x00\x02W/cf:cq/1697385871888/Put/vlen=4/seqid=0; \x00\x00\x02X/cf:cq/1697385871889/Put/vlen=4/seqid=0; \x00\x00\x02Y/cf:cq/1697385871891/Put/vlen=4/seqid=0; \x00\x00\x02Z/cf:cq/1697385871892/Put/vlen=4/seqid=0; \x00\x00\x02[/cf:cq/1697385871894/Put/vlen=4/seqid=0; \x00\x00\x02\x5C/cf:cq/1697385871895/Put/vlen=4/seqid=0; \x00\x00\x02]/cf:cq/1697385871896/Put/vlen=4/seqid=0; \x00\x00\x02^/cf:cq/1697385871898/Put/vlen=4/seqid=0; \x00\x00\x02_/cf:cq/1697385871899/Put/vlen=4/seqid=0; \x00\x00\x02`/cf:cq/1697385871900/Put/vlen=4/seqid=0; \x00\x00\x02a/cf:cq/1697385871901/Put/vlen=4/seqid=0; \x00\x00\x02b/cf:cq/1697385871903/Put/vlen=4/seqid=0; \x00\x00\x02c/cf:cq/1697385871904/Put/vlen=4/seqid=0; \x00\x00\x02d/cf:cq/1697385871905/Put/vlen=4/seqid=0; \x00\x00\x02e/cf:cq/1697385871908/Put/vlen=4/seqid=0; \x00\x00\x02f/cf:cq/1697385871910/Put/vlen=4/seqid=0; \x00\x00\x02g/cf:cq/1697385871912/Put/vlen=4/seqid=0; \x00\x00\x02h/cf:cq/1697385871913/Put/vlen=4/seqid=0; \x00\x00\x02i/cf:cq/1697385871915/Put/vlen=4/seqid=0; \x00\x00\x02j/cf:cq/1697385871916/Put/vlen=4/seqid=0; \x00\x00\x02k/cf:cq/1697385871918/Put/vlen=4/seqid=0; \x00\x00\x02l/cf:cq/1697385871920/Put/vlen=4/seqid=0; \x00\x00\x02m/cf:cq/1697385871921/Put/vlen=4/seqid=0; \x00\x00\x02n/cf:cq/1697385871923/Put/vlen=4/seqid=0; \x00\x00\x02o/cf:cq/1697385871924/Put/vlen=4/seqid=0; \x00\x00\x02p/cf:cq/1697385871925/Put/vlen=4/seqid=0; \x00\x00\x02q/cf:cq/1697385871927/Put/vlen=4/seqid=0; \x00\x00\x02r/cf:cq/1697385871928/Put/vlen=4/seqid=0; \x00\x00\x02s/cf:cq/1697385871929/Put/vlen=4/seqid=0; \x00\x00\x02t/cf:cq/1697385871930/Put/vlen=4/seqid=0; \x00\x00\x02u/cf:cq/1697385871932/Put/vlen=4/seqid=0; \x00\x00\x02v/cf:cq/1697385871933/Put/vlen=4/seqid=0; \x00\x00\x02w/cf:cq/1697385871934/Put/vlen=4/seqid=0; \x00\x00\x02x/cf:cq/1697385871935/Put/vlen=4/seqid=0; \x00\x00\x02y/cf:cq/1697385871937/Put/vlen=4/seqid=0; \x00\x00\x02z/cf:cq/1697385871938/Put/vlen=4/seqid=0; \x00\x00\x02{/cf:cq/1697385871939/Put/vlen=4/seqid=0; \x00\x00\x02|/cf:cq/1697385871940/Put/vlen=4/seqid=0; \x00\x00\x02}/cf:cq/1697385871941/Put/vlen=4/seqid=0; \x00\x00\x02~/cf:cq/1697385871942/Put/vlen=4/seqid=0; \x00\x00\x02\x7F/cf:cq/1697385871943/Put/vlen=4/seqid=0; \x00\x00\x02\x80/cf:cq/1697385871945/Put/vlen=4/seqid=0; \x00\x00\x02\x81/cf:cq/1697385871946/Put/vlen=4/seqid=0; \x00\x00\x02\x82/cf:cq/1697385871947/Put/vlen=4/seqid=0; \x00\x00\x02\x83/cf:cq/1697385871948/Put/vlen=4/seqid=0; \x00\x00\x02\x84/cf:cq/1697385871950/Put/vlen=4/seqid=0; \x00\x00\x02\x85/cf:cq/1697385871951/Put/vlen=4/seqid=0; \x00\x00\x02\x86/cf:cq/1697385871952/Put/vlen=4/seqid=0; \x00\x00\x02\x87/cf:cq/1697385871954/Put/vlen=4/seqid=0; \x00\x00\x02\x88/cf:cq/1697385871956/Put/vlen=4/seqid=0; \x00\x00\x02\x89/cf:cq/1697385871957/Put/vlen=4/seqid=0; \x00\x00\x02\x8A/cf:cq/1697385871958/Put/vlen=4/seqid=0; \x00\x00\x02\x8B/cf:cq/1697385871960/Put/vlen=4/seqid=0; \x00\x00\x02\x8C/cf:cq/1697385871962/Put/vlen=4/seqid=0; \x00\x00\x02\x8D/cf:cq/1697385871963/Put/vlen=4/seqid=0; \x00\x00\x02\x8E/cf:cq/1697385871965/Put/vlen=4/seqid=0; \x00\x00\x02\x8F/cf:cq/1697385871967/Put/vlen=4/seqid=0; \x00\x00\x02\x90/cf:cq/1697385871969/Put/vlen=4/seqid=0; \x00\x00\x02\x91/cf:cq/1697385871971/Put/vlen=4/seqid=0; \x00\x00\x02\x92/cf:cq/1697385871972/Put/vlen=4/seqid=0; \x00\x00\x02\x93/cf:cq/1697385871974/Put/vlen=4/seqid=0; \x00\x00\x02\x94/cf:cq/1697385871976/Put/vlen=4/seqid=0; \x00\x00\x02\x95/cf:cq/1697385871978/Put/vlen=4/seqid=0; \x00\x00\x02\x96/cf:cq/1697385871979/Put/vlen=4/seqid=0; \x00\x00\x02\x97/cf:cq/1697385871981/Put/vlen=4/seqid=0; \x00\x00\x02\x98/cf:cq/1697385871983/Put/vlen=4/seqid=0; \x00\x00\x02\x99/cf:cq/1697385871984/Put/vlen=4/seqid=0; \x00\x00\x02\x9A/cf:cq/1697385871986/Put/vlen=4/seqid=0; \x00\x00\x02\x9B/cf:cq/1697385871987/Put/vlen=4/seqid=0; \x00\x00\x02\x9C/cf:cq/1697385871989/Put/vlen=4/seqid=0; \x00\x00\x02\x9D/cf:cq/1697385871991/Put/vlen=4/seqid=0; \x00\x00\x02\x9E/cf:cq/1697385871993/Put/vlen=4/seqid=0; \x00\x00\x02\x9F/cf:cq/1697385871994/Put/vlen=4/seqid=0; \x00\x00\x02\xA0/cf:cq/1697385871996/Put/vlen=4/seqid=0; \x00\x00\x02\xA1/cf:cq/1697385871998/Put/vlen=4/seqid=0; \x00\x00\x02\xA2/cf:cq/1697385872000/Put/vlen=4/seqid=0; \x00\x00\x02\xA3/cf:cq/1697385872001/Put/vlen=4/seqid=0; \x00\x00\x02\xA4/cf:cq/1697385872003/Put/vlen=4/seqid=0; \x00\x00\x02\xA5/cf:cq/1697385872005/Put/vlen=4/seqid=0; \x00\x00\x02\xA6/cf:cq/1697385872007/Put/vlen=4/seqid=0; \x00\x00\x02\xA7/cf:cq/1697385872008/Put/vlen=4/seqid=0; \x00\x00\x02\xA8/cf:cq/1697385872010/Put/vlen=4/seqid=0; \x00\x00\x02\xA9/cf:cq/1697385872012/Put/vlen=4/seqid=0; \x00\x00\x02\xAA/cf:cq/1697385872013/Put/vlen=4/seqid=0; \x00\x00\x02\xAB/cf:cq/1697385872015/Put/vlen=4/seqid=0; \x00\x00\x02\xAC/cf:cq/1697385872016/Put/vlen=4/seqid=0; \x00\x00\x02\xAD/cf:cq/1697385872017/Put/vlen=4/seqid=0; \x00\x00\x02\xAE/cf:cq/1697385872018/Put/vlen=4/seqid=0; \x00\x00\x02\xAF/cf:cq/1697385872019/Put/vlen=4/seqid=0; \x00\x00\x02\xB0/cf:cq/1697385872022/Put/vlen=4/seqid=0; \x00\x00\x02\xB1/cf:cq/1697385872023/Put/vlen=4/seqid=0; \x00\x00\x02\xB2/cf:cq/1697385872025/Put/vlen=4/seqid=0; \x00\x00\x02\xB3/cf:cq/1697385872026/Put/vlen=4/seqid=0; \x00\x00\x02\xB4/cf:cq/1697385872028/Put/vlen=4/seqid=0; \x00\x00\x02\xB5/cf:cq/1697385872030/Put/vlen=4/seqid=0; \x00\x00\x02\xB6/cf:cq/1697385872032/Put/vlen=4/seqid=0; \x00\x00\x02\xB7/cf:cq/1697385872034/Put/vlen=4/seqid=0; \x00\x00\x02\xB8/cf:cq/1697385872036/Put/vlen=4/seqid=0; \x00\x00\x02\xB9/cf:cq/1697385872053/Put/vlen=4/seqid=0; \x00\x00\x02\xBA/cf:cq/1697385872055/Put/vlen=4/seqid=0; \x00\x00\x02\xBB/cf:cq/1697385872056/Put/vlen=4/seqid=0; \x00\x00\x02\xBC/cf:cq/1697385872058/Put/vlen=4/seqid=0; \x00\x00\x02\xBD/cf:cq/1697385872060/Put/vlen=4/seqid=0; \x00\x00\x02\xBE/cf:cq/1697385872061/Put/vlen=4/seqid=0; \x00\x00\x02\xBF/cf:cq/1697385872062/Put/vlen=4/seqid=0; \x00\x00\x02\xC0/cf:cq/1697385872064/Put/vlen=4/seqid=0; \x00\x00\x02\xC1/cf:cq/1697385872066/Put/vlen=4/seqid=0; \x00\x00\x02\xC2/cf:cq/1697385872067/Put/vlen=4/seqid=0; \x00\x00\x02\xC3/cf:cq/1697385872068/Put/vlen=4/seqid=0; \x00\x00\x02\xC4/cf:cq/1697385872069/Put/vlen=4/seqid=0; \x00\x00\x02\xC5/cf:cq/1697385872071/Put/vlen=4/seqid=0; \x00\x00\x02\xC6/cf:cq/1697385872074/Put/vlen=4/seqid=0; \x00\x00\x02\xC7/cf:cq/1697385872075/Put/vlen=4/seqid=0; \x00\x00\x02\xC8/cf:cq/1697385872077/Put/vlen=4/seqid=0; \x00\x00\x02\xC9/cf:cq/1697385872078/Put/vlen=4/seqid=0; \x00\x00\x02\xCA/cf:cq/1697385872080/Put/vlen=4/seqid=0; \x00\x00\x02\xCB/cf:cq/1697385872081/Put/vlen=4/seqid=0; \x00\x00\x02\xCC/cf:cq/1697385872082/Put/vlen=4/seqid=0; \x00\x00\x02\xCD/cf:cq/1697385872084/Put/vlen=4/seqid=0; \x00\x00\x02\xCE/cf:cq/1697385872085/Put/vlen=4/seqid=0; \x00\x00\x02\xCF/cf:cq/1697385872087/Put/vlen=4/seqid=0; \x00\x00\x02\xD0/cf:cq/1697385872088/Put/vlen=4/seqid=0; \x00\x00\x02\xD1/cf:cq/1697385872090/Put/vlen=4/seqid=0; \x00\x00\x02\xD2/cf:cq/1697385872091/Put/vlen=4/seqid=0; \x00\x00\x02\xD3/cf:cq/1697385872093/Put/vlen=4/seqid=0; \x00\x00\x02\xD4/cf:cq/1697385872095/Put/vlen=4/seqid=0; \x00\x00\x02\xD5/cf:cq/1697385872097/Put/vlen=4/seqid=0; \x00\x00\x02\xD6/cf:cq/1697385872098/Put/vlen=4/seqid=0; \x00\x00\x02\xD7/cf:cq/1697385872100/Put/vlen=4/seqid=0; \x00\x00\x02\xD8/cf:cq/1697385872102/Put/vlen=4/seqid=0; \x00\x00\x02\xD9/cf:cq/1697385872104/Put/vlen=4/seqid=0; \x00\x00\x02\xDA/cf:cq/1697385872106/Put/vlen=4/seqid=0; \x00\x00\x02\xDB/cf:cq/1697385872107/Put/vlen=4/seqid=0; \x00\x00\x02\xDC/cf:cq/1697385872111/Put/vlen=4/seqid=0; \x00\x00\x02\xDD/cf:cq/1697385872114/Put/vlen=4/seqid=0; \x00\x00\x02\xDE/cf:cq/1697385872115/Put/vlen=4/seqid=0; \x00\x00\x02\xDF/cf:cq/1697385872117/Put/vlen=4/seqid=0; \x00\x00\x02\xE0/cf:cq/1697385872118/Put/vlen=4/seqid=0; \x00\x00\x02\xE1/cf:cq/1697385872119/Put/vlen=4/seqid=0; \x00\x00\x02\xE2/cf:cq/1697385872121/Put/vlen=4/seqid=0; \x00\x00\x02\xE3/cf:cq/1697385872122/Put/vlen=4/seqid=0; \x00\x00\x02\xE4/cf:cq/1697385872124/Put/vlen=4/seqid=0; \x00\x00\x02\xE5/cf:cq/1697385872125/Put/vlen=4/seqid=0; \x00\x00\x02\xE6/cf:cq/1697385872127/Put/vlen=4/seqid=0; \x00\x00\x02\xE7/cf:cq/1697385872128/Put/vlen=4/seqid=0; \x00\x00\x02\xE8/cf:cq/1697385872129/Put/vlen=4/seqid=0; \x00\x00\x02\xE9/cf:cq/1697385872131/Put/vlen=4/seqid=0; \x00\x00\x02\xEA/cf:cq/1697385872132/Put/vlen=4/seqid=0; \x00\x00\x02\xEB/cf:cq/1697385872134/Put/vlen=4/seqid=0; \x00\x00\x02\xEC/cf:cq/1697385872135/Put/vlen=4/seqid=0; \x00\x00\x02\xED/cf:cq/1697385872138/Put/vlen=4/seqid=0; \x00\x00\x02\xEE/cf:cq/1697385872139/Put/vlen=4/seqid=0; \x00\x00\x02\xEF/cf:cq/1697385872140/Put/vlen=4/seqid=0; \x00\x00\x02\xF0/cf:cq/1697385872142/Put/vlen=4/seqid=0; \x00\x00\x02\xF1/cf:cq/1697385872143/Put/vlen=4/seqid=0; \x00\x00\x02\xF2/cf:cq/1697385872144/Put/vlen=4/seqid=0; \x00\x00\x02\xF3/cf:cq/1697385872146/Put/vlen=4/seqid=0; \x00\x00\x02\xF4/cf:cq/1697385872147/Put/vlen=4/seqid=0; \x00\x00\x02\xF5/cf:cq/1697385872149/Put/vlen=4/seqid=0; \x00\x00\x02\xF6/cf:cq/1697385872150/Put/vlen=4/seqid=0; \x00\x00\x02\xF7/cf:cq/1697385872152/Put/vlen=4/seqid=0; \x00\x00\x02\xF8/cf:cq/1697385872153/Put/vlen=4/seqid=0; \x00\x00\x02\xF9/cf:cq/1697385872155/Put/vlen=4/seqid=0; \x00\x00\x02\xFA/cf:cq/1697385872156/Put/vlen=4/seqid=0; \x00\x00\x02\xFB/cf:cq/1697385872157/Put/vlen=4/seqid=0; \x00\x00\x02\xFC/cf:cq/1697385872158/Put/vlen=4/seqid=0; \x00\x00\x02\xFD/cf:cq/1697385872160/Put/vlen=4/seqid=0; \x00\x00\x02\xFE/cf:cq/1697385872162/Put/vlen=4/seqid=0; \x00\x00\x02\xFF/cf:cq/1697385872164/Put/vlen=4/seqid=0; \x00\x00\x03\x00/cf:cq/1697385872166/Put/vlen=4/seqid=0; \x00\x00\x03\x01/cf:cq/1697385872167/Put/vlen=4/seqid=0; \x00\x00\x03\x02/cf:cq/1697385872169/Put/vlen=4/seqid=0; \x00\x00\x03\x03/cf:cq/1697385872170/Put/vlen=4/seqid=0; \x00\x00\x03\x04/cf:cq/1697385872171/Put/vlen=4/seqid=0; \x00\x00\x03\x05/cf:cq/1697385872173/Put/vlen=4/seqid=0; \x00\x00\x03\x06/cf:cq/1697385872175/Put/vlen=4/seqid=0; \x00\x00\x03\x07/cf:cq/1697385872176/Put/vlen=4/seqid=0; \x00\x00\x03\x08/cf:cq/1697385872178/Put/vlen=4/seqid=0; \x00\x00\x03\x09/cf:cq/1697385872179/Put/vlen=4/seqid=0; \x00\x00\x03\x0A/cf:cq/1697385872180/Put/vlen=4/seqid=0; \x00\x00\x03\x0B/cf:cq/1697385872181/Put/vlen=4/seqid=0; \x00\x00\x03\x0C/cf:cq/1697385872182/Put/vlen=4/seqid=0; \x00\x00\x03\x0D/cf:cq/1697385872184/Put/vlen=4/seqid=0; \x00\x00\x03\x0E/cf:cq/1697385872185/Put/vlen=4/seqid=0; \x00\x00\x03\x0F/cf:cq/1697385872186/Put/vlen=4/seqid=0; \x00\x00\x03\x10/cf:cq/1697385872187/Put/vlen=4/seqid=0; \x00\x00\x03\x11/cf:cq/1697385872188/Put/vlen=4/seqid=0; \x00\x00\x03\x12/cf:cq/1697385872190/Put/vlen=4/seqid=0; \x00\x00\x03\x13/cf:cq/1697385872191/Put/vlen=4/seqid=0; \x00\x00\x03\x14/cf:cq/1697385872194/Put/vlen=4/seqid=0; \x00\x00\x03\x15/cf:cq/1697385872195/Put/vlen=4/seqid=0; \x00\x00\x03\x16/cf:cq/1697385872197/Put/vlen=4/seqid=0; \x00\x00\x03\x17/cf:cq/1697385872198/Put/vlen=4/seqid=0; \x00\x00\x03\x18/cf:cq/1697385872200/Put/vlen=4/seqid=0; \x00\x00\x03\x19/cf:cq/1697385872204/Put/vlen=4/seqid=0; \x00\x00\x03\x1A/cf:cq/1697385872207/Put/vlen=4/seqid=0; \x00\x00\x03\x1B/cf:cq/1697385872208/Put/vlen=4/seqid=0; \x00\x00\x03\x1C/cf:cq/1697385872210/Put/vlen=4/seqid=0; \x00\x00\x03\x1D/cf:cq/1697385872211/Put/vlen=4/seqid=0; \x00\x00\x03\x1E/cf:cq/1697385872212/Put/vlen=4/seqid=0; \x00\x00\x03\x1F/cf:cq/1697385872214/Put/vlen=4/seqid=0; \x00\x00\x03 /cf:cq/1697385872216/Put/vlen=4/seqid=0; \x00\x00\x03!/cf:cq/1697385872217/Put/vlen=4/seqid=0; \x00\x00\x03"/cf:cq/1697385872219/Put/vlen=4/seqid=0; \x00\x00\x03#/cf:cq/1697385872224/Put/vlen=4/seqid=0; \x00\x00\x03$/cf:cq/1697385872225/Put/vlen=4/seqid=0; \x00\x00\x03%/cf:cq/1697385872227/Put/vlen=4/seqid=0; \x00\x00\x03&/cf:cq/1697385872228/Put/vlen=4/seqid=0; \x00\x00\x03'/cf:cq/1697385872229/Put/vlen=4/seqid=0; \x00\x00\x03(/cf:cq/1697385872231/Put/vlen=4/seqid=0; \x00\x00\x03)/cf:cq/1697385872232/Put/vlen=4/seqid=0; \x00\x00\x03*/cf:cq/1697385872233/Put/vlen=4/seqid=0; \x00\x00\x03+/cf:cq/1697385872234/Put/vlen=4/seqid=0; \x00\x00\x03,/cf:cq/1697385872236/Put/vlen=4/seqid=0; \x00\x00\x03-/cf:cq/1697385872237/Put/vlen=4/seqid=0; \x00\x00\x03./cf:cq/1697385872240/Put/vlen=4/seqid=0; \x00\x00\x03//cf:cq/1697385872241/Put/vlen=4/seqid=0; \x00\x00\x030/cf:cq/1697385872242/Put/vlen=4/seqid=0; \x00\x00\x031/cf:cq/1697385872244/Put/vlen=4/seqid=0; \x00\x00\x032/cf:cq/1697385872245/Put/vlen=4/seqid=0; \x00\x00\x033/cf:cq/1697385872257/Put/vlen=4/seqid=0; \x00\x00\x034/cf:cq/1697385872258/Put/vlen=4/seqid=0; \x00\x00\x035/cf:cq/1697385872260/Put/vlen=4/seqid=0; \x00\x00\x036/cf:cq/1697385872261/Put/vlen=4/seqid=0; \x00\x00\x037/cf:cq/1697385872263/Put/vlen=4/seqid=0; \x00\x00\x038/cf:cq/1697385872264/Put/vlen=4/seqid=0; \x00\x00\x039/cf:cq/1697385872267/Put/vlen=4/seqid=0; \x00\x00\x03:/cf:cq/1697385872269/Put/vlen=4/seqid=0; \x00\x00\x03;/cf:cq/1697385872270/Put/vlen=4/seqid=0; \x00\x00\x03/cf:cq/1697385872276/Put/vlen=4/seqid=0; \x00\x00\x03?/cf:cq/1697385872279/Put/vlen=4/seqid=0; \x00\x00\x03@/cf:cq/1697385872283/Put/vlen=4/seqid=0; \x00\x00\x03A/cf:cq/1697385872285/Put/vlen=4/seqid=0; \x00\x00\x03B/cf:cq/1697385872287/Put/vlen=4/seqid=0; \x00\x00\x03C/cf:cq/1697385872289/Put/vlen=4/seqid=0; \x00\x00\x03D/cf:cq/1697385872297/Put/vlen=4/seqid=0; \x00\x00\x03E/cf:cq/1697385872299/Put/vlen=4/seqid=0; \x00\x00\x03F/cf:cq/1697385872302/Put/vlen=4/seqid=0; \x00\x00\x03G/cf:cq/1697385872312/Put/vlen=4/seqid=0; \x00\x00\x03H/cf:cq/1697385872315/Put/vlen=4/seqid=0; \x00\x00\x03I/cf:cq/1697385872319/Put/vlen=4/seqid=0; \x00\x00\x03J/cf:cq/1697385872321/Put/vlen=4/seqid=0; \x00\x00\x03K/cf:cq/1697385872322/Put/vlen=4/seqid=0; \x00\x00\x03L/cf:cq/1697385872323/Put/vlen=4/seqid=0; \x00\x00\x03M/cf:cq/1697385872325/Put/vlen=4/seqid=0; \x00\x00\x03N/cf:cq/1697385872327/Put/vlen=4/seqid=0; \x00\x00\x03O/cf:cq/1697385872328/Put/vlen=4/seqid=0; \x00\x00\x03P/cf:cq/1697385872329/Put/vlen=4/seqid=0; \x00\x00\x03Q/cf:cq/1697385872330/Put/vlen=4/seqid=0; \x00\x00\x03R/cf:cq/1697385872332/Put/vlen=4/seqid=0; \x00\x00\x03S/cf:cq/1697385872333/Put/vlen=4/seqid=0; \x00\x00\x03T/cf:cq/1697385872335/Put/vlen=4/seqid=0; \x00\x00\x03U/cf:cq/1697385872336/Put/vlen=4/seqid=0; \x00\x00\x03V/cf:cq/1697385872338/Put/vlen=4/seqid=0; \x00\x00\x03W/cf:cq/1697385872340/Put/vlen=4/seqid=0; \x00\x00\x03X/cf:cq/1697385872342/Put/vlen=4/seqid=0; \x00\x00\x03Y/cf:cq/1697385872343/Put/vlen=4/seqid=0; \x00\x00\x03Z/cf:cq/1697385872344/Put/vlen=4/seqid=0; \x00\x00\x03[/cf:cq/1697385872346/Put/vlen=4/seqid=0; \x00\x00\x03\x5C/cf:cq/1697385872347/Put/vlen=4/seqid=0; \x00\x00\x03]/cf:cq/1697385872348/Put/vlen=4/seqid=0; \x00\x00\x03^/cf:cq/1697385872350/Put/vlen=4/seqid=0; \x00\x00\x03_/cf:cq/1697385872351/Put/vlen=4/seqid=0; \x00\x00\x03`/cf:cq/1697385872352/Put/vlen=4/seqid=0; \x00\x00\x03a/cf:cq/1697385872354/Put/vlen=4/seqid=0; \x00\x00\x03b/cf:cq/1697385872355/Put/vlen=4/seqid=0; \x00\x00\x03c/cf:cq/1697385872356/Put/vlen=4/seqid=0; \x00\x00\x03d/cf:cq/1697385872357/Put/vlen=4/seqid=0; \x00\x00\x03e/cf:cq/1697385872358/Put/vlen=4/seqid=0; \x00\x00\x03f/cf:cq/1697385872359/Put/vlen=4/seqid=0; \x00\x00\x03g/cf:cq/1697385872361/Put/vlen=4/seqid=0; \x00\x00\x03h/cf:cq/1697385872362/Put/vlen=4/seqid=0; \x00\x00\x03i/cf:cq/1697385872364/Put/vlen=4/seqid=0; \x00\x00\x03j/cf:cq/1697385872366/Put/vlen=4/seqid=0; \x00\x00\x03k/cf:cq/1697385872367/Put/vlen=4/seqid=0; \x00\x00\x03l/cf:cq/1697385872369/Put/vlen=4/seqid=0; \x00\x00\x03m/cf:cq/1697385872370/Put/vlen=4/seqid=0; \x00\x00\x03n/cf:cq/1697385872371/Put/vlen=4/seqid=0; \x00\x00\x03o/cf:cq/1697385872372/Put/vlen=4/seqid=0; \x00\x00\x03p/cf:cq/1697385872373/Put/vlen=4/seqid=0; \x00\x00\x03q/cf:cq/1697385872374/Put/vlen=4/seqid=0; \x00\x00\x03r/cf:cq/1697385872375/Put/vlen=4/seqid=0; \x00\x00\x03s/cf:cq/1697385872377/Put/vlen=4/seqid=0; \x00\x00\x03t/cf:cq/1697385872378/Put/vlen=4/seqid=0; \x00\x00\x03u/cf:cq/1697385872379/Put/vlen=4/seqid=0; \x00\x00\x03v/cf:cq/1697385872380/Put/vlen=4/seqid=0; \x00\x00\x03w/cf:cq/1697385872381/Put/vlen=4/seqid=0; \x00\x00\x03x/cf:cq/1697385872382/Put/vlen=4/seqid=0; \x00\x00\x03y/cf:cq/1697385872383/Put/vlen=4/seqid=0; \x00\x00\x03z/cf:cq/1697385872384/Put/vlen=4/seqid=0; \x00\x00\x03{/cf:cq/1697385872385/Put/vlen=4/seqid=0; \x00\x00\x03|/cf:cq/1697385872386/Put/vlen=4/seqid=0; \x00\x00\x03}/cf:cq/1697385872388/Put/vlen=4/seqid=0; \x00\x00\x03~/cf:cq/1697385872389/Put/vlen=4/seqid=0; \x00\x00\x03\x7F/cf:cq/1697385872390/Put/vlen=4/seqid=0; \x00\x00\x03\x80/cf:cq/1697385872391/Put/vlen=4/seqid=0; \x00\x00\x03\x81/cf:cq/1697385872392/Put/vlen=4/seqid=0; \x00\x00\x03\x82/cf:cq/1697385872393/Put/vlen=4/seqid=0; \x00\x00\x03\x83/cf:cq/1697385872394/Put/vlen=4/seqid=0; \x00\x00\x03\x84/cf:cq/1697385872395/Put/vlen=4/seqid=0; \x00\x00\x03\x85/cf:cq/1697385872396/Put/vlen=4/seqid=0; \x00\x00\x03\x86/cf:cq/1697385872397/Put/vlen=4/seqid=0; \x00\x00\x03\x87/cf:cq/1697385872399/Put/vlen=4/seqid=0; \x00\x00\x03\x88/cf:cq/1697385872400/Put/vlen=4/seqid=0; \x00\x00\x03\x89/cf:cq/1697385872401/Put/vlen=4/seqid=0; \x00\x00\x03\x8A/cf:cq/1697385872402/Put/vlen=4/seqid=0; \x00\x00\x03\x8B/cf:cq/1697385872403/Put/vlen=4/seqid=0; \x00\x00\x03\x8C/cf:cq/1697385872405/Put/vlen=4/seqid=0; \x00\x00\x03\x8D/cf:cq/1697385872406/Put/vlen=4/seqid=0; \x00\x00\x03\x8E/cf:cq/1697385872407/Put/vlen=4/seqid=0; \x00\x00\x03\x8F/cf:cq/1697385872408/Put/vlen=4/seqid=0; \x00\x00\x03\x90/cf:cq/1697385872409/Put/vlen=4/seqid=0; \x00\x00\x03\x91/cf:cq/1697385872411/Put/vlen=4/seqid=0; \x00\x00\x03\x92/cf:cq/1697385872412/Put/vlen=4/seqid=0; \x00\x00\x03\x93/cf:cq/1697385872413/Put/vlen=4/seqid=0; \x00\x00\x03\x94/cf:cq/1697385872415/Put/vlen=4/seqid=0; \x00\x00\x03\x95/cf:cq/1697385872416/Put/vlen=4/seqid=0; \x00\x00\x03\x96/cf:cq/1697385872417/Put/vlen=4/seqid=0; \x00\x00\x03\x97/cf:cq/1697385872420/Put/vlen=4/seqid=0; \x00\x00\x03\x98/cf:cq/1697385872421/Put/vlen=4/seqid=0; \x00\x00\x03\x99/cf:cq/1697385872422/Put/vlen=4/seqid=0; \x00\x00\x03\x9A/cf:cq/1697385872423/Put/vlen=4/seqid=0; \x00\x00\x03\x9B/cf:cq/1697385872425/Put/vlen=4/seqid=0; \x00\x00\x03\x9C/cf:cq/1697385872426/Put/vlen=4/seqid=0; \x00\x00\x03\x9D/cf:cq/1697385872427/Put/vlen=4/seqid=0; \x00\x00\x03\x9E/cf:cq/1697385872428/Put/vlen=4/seqid=0; \x00\x00\x03\x9F/cf:cq/1697385872429/Put/vlen=4/seqid=0; \x00\x00\x03\xA0/cf:cq/1697385872430/Put/vlen=4/seqid=0; \x00\x00\x03\xA1/cf:cq/1697385872431/Put/vlen=4/seqid=0; \x00\x00\x03\xA2/cf:cq/1697385872432/Put/vlen=4/seqid=0; \x00\x00\x03\xA3/cf:cq/1697385872433/Put/vlen=4/seqid=0; \x00\x00\x03\xA4/cf:cq/1697385872435/Put/vlen=4/seqid=0; \x00\x00\x03\xA5/cf:cq/1697385872436/Put/vlen=4/seqid=0; \x00\x00\x03\xA6/cf:cq/1697385872437/Put/vlen=4/seqid=0; \x00\x00\x03\xA7/cf:cq/1697385872438/Put/vlen=4/seqid=0; \x00\x00\x03\xA8/cf:cq/1697385872439/Put/vlen=4/seqid=0; \x00\x00\x03\xA9/cf:cq/1697385872440/Put/vlen=4/seqid=0; \x00\x00\x03\xAA/cf:cq/1697385872442/Put/vlen=4/seqid=0; \x00\x00\x03\xAB/cf:cq/1697385872443/Put/vlen=4/seqid=0; \x00\x00\x03\xAC/cf:cq/1697385872445/Put/vlen=4/seqid=0; \x00\x00\x03\xAD/cf:cq/1697385872446/Put/vlen=4/seqid=0; \x00\x00\x03\xAE/cf:cq/1697385872447/Put/vlen=4/seqid=0; \x00\x00\x03\xAF/cf:cq/1697385872448/Put/vlen=4/seqid=0; \x00\x00\x03\xB0/cf:cq/1697385872449/Put/vlen=4/seqid=0; \x00\x00\x03\xB1/cf:cq/1697385872450/Put/vlen=4/seqid=0; \x00\x00\x03\xB2/cf:cq/1697385872451/Put/vlen=4/seqid=0; \x00\x00\x03\xB3/cf:cq/1697385872453/Put/vlen=4/seqid=0; \x00\x00\x03\xB4/cf:cq/1697385872454/Put/vlen=4/seqid=0; \x00\x00\x03\xB5/cf:cq/1697385872455/Put/vlen=4/seqid=0; \x00\x00\x03\xB6/cf:cq/1697385872456/Put/vlen=4/seqid=0; \x00\x00\x03\xB7/cf:cq/1697385872458/Put/vlen=4/seqid=0; \x00\x00\x03\xB8/cf:cq/1697385872459/Put/vlen=4/seqid=0; \x00\x00\x03\xB9/cf:cq/1697385872460/Put/vlen=4/seqid=0; \x00\x00\x03\xBA/cf:cq/1697385872461/Put/vlen=4/seqid=0; \x00\x00\x03\xBB/cf:cq/1697385872462/Put/vlen=4/seqid=0; \x00\x00\x03\xBC/cf:cq/1697385872464/Put/vlen=4/seqid=0; \x00\x00\x03\xBD/cf:cq/1697385872465/Put/vlen=4/seqid=0; \x00\x00\x03\xBE/cf:cq/1697385872466/Put/vlen=4/seqid=0; \x00\x00\x03\xBF/cf:cq/1697385872467/Put/vlen=4/seqid=0; \x00\x00\x03\xC0/cf:cq/1697385872468/Put/vlen=4/seqid=0; \x00\x00\x03\xC1/cf:cq/1697385872474/Put/vlen=4/seqid=0; \x00\x00\x03\xC2/cf:cq/1697385872478/Put/vlen=4/seqid=0; \x00\x00\x03\xC3/cf:cq/1697385872482/Put/vlen=4/seqid=0; \x00\x00\x03\xC4/cf:cq/1697385872484/Put/vlen=4/seqid=0; \x00\x00\x03\xC5/cf:cq/1697385872489/Put/vlen=4/seqid=0; \x00\x00\x03\xC6/cf:cq/1697385872491/Put/vlen=4/seqid=0; \x00\x00\x03\xC7/cf:cq/1697385872494/Put/vlen=4/seqid=0; \x00\x00\x03\xC8/cf:cq/1697385872495/Put/vlen=4/seqid=0; \x00\x00\x03\xC9/cf:cq/1697385872497/Put/vlen=4/seqid=0; \x00\x00\x03\xCA/cf:cq/1697385872499/Put/vlen=4/seqid=0; \x00\x00\x03\xCB/cf:cq/1697385872501/Put/vlen=4/seqid=0; \x00\x00\x03\xCC/cf:cq/1697385872503/Put/vlen=4/seqid=0; \x00\x00\x03\xCD/cf:cq/1697385872504/Put/vlen=4/seqid=0; \x00\x00\x03\xCE/cf:cq/1697385872506/Put/vlen=4/seqid=0; \x00\x00\x03\xCF/cf:cq/1697385872508/Put/vlen=4/seqid=0; \x00\x00\x03\xD0/cf:cq/1697385872509/Put/vlen=4/seqid=0; \x00\x00\x03\xD1/cf:cq/1697385872514/Put/vlen=4/seqid=0; \x00\x00\x03\xD2/cf:cq/1697385872517/Put/vlen=4/seqid=0; \x00\x00\x03\xD3/cf:cq/1697385872518/Put/vlen=4/seqid=0; \x00\x00\x03\xD4/cf:cq/1697385872520/Put/vlen=4/seqid=0; \x00\x00\x03\xD5/cf:cq/1697385872524/Put/vlen=4/seqid=0; \x00\x00\x03\xD6/cf:cq/1697385872525/Put/vlen=4/seqid=0; \x00\x00\x03\xD7/cf:cq/1697385872526/Put/vlen=4/seqid=0; \x00\x00\x03\xD8/cf:cq/1697385872528/Put/vlen=4/seqid=0; \x00\x00\x03\xD9/cf:cq/1697385872529/Put/vlen=4/seqid=0; \x00\x00\x03\xDA/cf:cq/1697385872530/Put/vlen=4/seqid=0; \x00\x00\x03\xDB/cf:cq/1697385872531/Put/vlen=4/seqid=0; \x00\x00\x03\xDC/cf:cq/1697385872532/Put/vlen=4/seqid=0; \x00\x00\x03\xDD/cf:cq/1697385872533/Put/vlen=4/seqid=0; \x00\x00\x03\xDE/cf:cq/1697385872534/Put/vlen=4/seqid=0; \x00\x00\x03\xDF/cf:cq/1697385872536/Put/vlen=4/seqid=0; \x00\x00\x03\xE0/cf:cq/1697385872537/Put/vlen=4/seqid=0; \x00\x00\x03\xE1/cf:cq/1697385872538/Put/vlen=4/seqid=0; \x00\x00\x03\xE2/cf:cq/1697385872540/Put/vlen=4/seqid=0; \x00\x00\x03\xE3/cf:cq/1697385872542/Put/vlen=4/seqid=0; \x00\x00\x03\xE4/cf:cq/1697385872544/Put/vlen=4/seqid=0; \x00\x00\x03\xE5/cf:cq/1697385872546/Put/vlen=4/seqid=0; \x00\x00\x03\xE6/cf:cq/1697385872547/Put/vlen=4/seqid=0; \x00\x00\x03\xE7/cf:cq/1697385872548/Put/vlen=4/seqid=0; >],80117}], lastWalPath=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C45999%2C1697385859285.1697385867224, lastWalPosition=36149, nbRowKeys=1000, nbHFiles=0, heapSize=80117, lastSeqIds={}, endOfFile=true,usedBufferSize=80117] 2023-10-15T16:04:44,949 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C45999%2C1697385859285.rep 2023-10-15T16:04:44,949 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:44,951 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:44,955 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(451): EOF, closing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525 2023-10-15T16:04:44,955 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceWALReader(175): Read 0 WAL entries eligible for replication 2023-10-15T16:04:44,955 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.WALEntryStream(236): No more WAL files in queue 2023-10-15T16:04:44,955 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525, lastWalPosition=1253, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-10-15T16:04:44,955 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.wal-reader.8c5d72df62de%2C45999%2C1697385859285.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceWALReader(118): Stopping the replication source wal reader 2023-10-15T16:04:44,959 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceManager(720): Removing 1 logs in the list: [8c5d72df62de%2C45999%2C1697385859285.rep.1697385874525] 2023-10-15T16:04:44,960 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:44,960 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceManager(734): Removing 0 logs from remote dir hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/remoteWALs in the list: [] 2023-10-15T16:04:44,961 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-10-15T16:04:44,961 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.ReplicationSourceShipper(138): Finished recovering queue for group 8c5d72df62de%2C45999%2C1697385859285.rep of peer 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 2023-10-15T16:04:45,050 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:45,070 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46527 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.ClaimReplicationQueueCallable, pid=55 2023-10-15T16:04:45,250 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:45,551 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:45,865 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:45,869 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:45,898 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:45,940 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=80117B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C46527%2C1697385859222]: currently replicating from: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 at position: 383 Recovered source for cluster/machine(s) 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C45999%2C1697385859285]: no replication ongoing, waiting for new log 2023-10-15T16:04:45,952 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:46,453 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:46,874 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:46,881 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:46,899 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:47,053 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:47,754 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:47,892 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:47,897 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:47,899 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,38435,1697385859175 still being processed, waiting 2023-10-15T16:04:48,129 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=55}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:48,130 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=55}] wal.AbstractFSWALProvider(366): Seprated old WAL dir hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de,38435,1697385859175 not exists 2023-10-15T16:04:48,130 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=55}] regionserver.ReplicationSourceManager(843): Skip enqueuing log 8c5d72df62de%2C38435%2C1697385859175.rep.1697385865402 because it is before the start offset 8c5d72df62de%2C38435%2C1697385859175.rep.1697385865402:-1 2023-10-15T16:04:48,130 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=55}] regionserver.ReplicationSourceManager(888): Added source for recovered queue 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175, number of wals to replicate: 3 2023-10-15T16:04:48,130 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=55}] regionserver.ReplicationSourceManager(891): Enqueueing log hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.1697385861137 from recovered queue for source: 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:48,130 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=55}] regionserver.ReplicationSourceManager(891): Enqueueing log hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.1697385867465 from recovered queue for source: 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:48,130 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=55}] regionserver.ReplicationSourceManager(891): Enqueueing log hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385867425 from recovered queue for source: 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:48,131 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=55}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=55 2023-10-15T16:04:48,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=55 2023-10-15T16:04:48,134 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=55, resume processing ppid=53 2023-10-15T16:04:48,134 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=55, ppid=53, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.ClaimReplicationQueueRemoteProcedure in 3.2170 sec 2023-10-15T16:04:48,142 DEBUG [PEWorker-4 {}] replication.AssignReplicationQueuesProcedure(116): Finish claiming replication queues for 8c5d72df62de,45999,1697385859285 2023-10-15T16:04:48,145 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=53, resume processing ppid=38 2023-10-15T16:04:48,145 INFO [PEWorker-1 {}] procedure.ServerCrashProcedure(285): removed crashed server 8c5d72df62de,45999,1697385859285 after splitting done 2023-10-15T16:04:48,145 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=53, ppid=38, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure in 7.0290 sec 2023-10-15T16:04:48,147 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:48,149 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x0072515b to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:48,149 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=38, state=SUCCESS, hasLock=false; ServerCrashProcedure 8c5d72df62de,45999,1697385859285, splitWal=true, meta=true in 8.6690 sec 2023-10-15T16:04:48,151 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:48,152 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f51002e connected 2023-10-15T16:04:48,155 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@69a43615, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:48,157 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 (queues=2) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:48,157 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C38435%2C1697385859175 2023-10-15T16:04:48,157 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:48,158 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.1697385861137, startPosition=0, beingWritten=false 2023-10-15T16:04:48,160 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C38435%2C1697385859175.rep 2023-10-15T16:04:48,160 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C38435%2C1697385859175 2023-10-15T16:04:48,160 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:48,160 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(451): EOF, closing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.1697385861137 2023-10-15T16:04:48,160 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C38435%2C1697385859175.rep 2023-10-15T16:04:48,161 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385867425, startPosition=0, beingWritten=false 2023-10-15T16:04:48,161 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.1697385867465, startPosition=0, beingWritten=false 2023-10-15T16:04:48,163 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(451): EOF, closing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.1697385867465 2023-10-15T16:04:48,163 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(236): No more WAL files in queue 2023-10-15T16:04:48,163 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(118): Stopping the replication source wal reader 2023-10-15T16:04:48,163 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-10-15T16:04:48,165 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(138): Finished recovering queue for group 8c5d72df62de%2C38435%2C1697385859175 of peer 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:48,166 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(451): EOF, closing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385867425 2023-10-15T16:04:48,166 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(175): Read 0 WAL entries eligible for replication 2023-10-15T16:04:48,166 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.WALEntryStream(236): No more WAL files in queue 2023-10-15T16:04:48,166 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceWALReader(118): Stopping the replication source wal reader 2023-10-15T16:04:48,166 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C38435%2C1697385859175.rep.1697385867425, lastWalPosition=1096, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-10-15T16:04:48,168 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceManager(720): Removing 1 logs in the list: [8c5d72df62de%2C38435%2C1697385859175.rep.1697385867425] 2023-10-15T16:04:48,169 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceManager(734): Removing 0 logs from remote dir hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/remoteWALs in the list: [] 2023-10-15T16:04:48,169 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-10-15T16:04:48,169 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceShipper(138): Finished recovering queue for group 8c5d72df62de%2C38435%2C1697385859175.rep of peer 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:48,169 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceManager(540): Done with the recovered queue 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 2023-10-15T16:04:48,176 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175.replicationSource.shipper8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 {}] regionserver.ReplicationSourceManager(553): Finished recovering queue 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175 with the following stats: Total replicated edits: 0, current progress: 2023-10-15T16:04:48,555 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:48,750 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:04:48,899 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(139): Server 8c5d72df62de,38435,1697385859175 done with server shutdown processing 2023-10-15T16:04:48,899 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(83): Going to start a new RS 2023-10-15T16:04:48,902 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323 to pos 383, reset compression=false 2023-10-15T16:04:48,906 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=true 2023-10-15T16:04:48,916 INFO [RegionServer-Restarter {}] client.ConnectionUtils(126): regionserver/8c5d72df62de:0 server-side Connection retries=45 2023-10-15T16:04:48,916 INFO [RegionServer-Restarter {}] ipc.RpcExecutor(188): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-10-15T16:04:48,916 INFO [RegionServer-Restarter {}] ipc.RpcExecutor(188): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-10-15T16:04:48,916 INFO [RegionServer-Restarter {}] ipc.RWQueueRpcExecutor(113): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-10-15T16:04:48,916 INFO [RegionServer-Restarter {}] ipc.RpcExecutor(188): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-10-15T16:04:48,916 INFO [RegionServer-Restarter {}] ipc.RpcExecutor(188): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-10-15T16:04:48,916 INFO [RegionServer-Restarter {}] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService, hbase.pb.ClientMetaService 2023-10-15T16:04:48,916 INFO [RegionServer-Restarter {}] ipc.NettyRpcServer(304): Using org.apache.hbase.thirdparty.io.netty.buffer.PooledByteBufAllocator for buffer allocation 2023-10-15T16:04:48,917 INFO [RegionServer-Restarter {}] ipc.NettyRpcServer(186): Bind to /172.17.0.3:35419 2023-10-15T16:04:48,917 INFO [RegionServer-Restarter {}] zookeeper.RecoverableZooKeeper(141): Process identifier=regionserver:35419 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:48,918 INFO [RegionServer-Restarter {}] fs.HFileSystem(339): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-10-15T16:04:48,920 DEBUG [RegionServer-Restarter-EventThread {}] zookeeper.ZKWatcher(604): regionserver:354190x0, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:48,920 INFO [RegionServer-Restarter {}] fs.HFileSystem(339): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-10-15T16:04:48,921 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): regionserver:35419-0x101b6f97f510030 connected 2023-10-15T16:04:48,927 DEBUG [RegionServer-Restarter {}] zookeeper.ZKUtil(111): regionserver:35419-0x101b6f97f510030, quorum=127.0.0.1:52753, baseZNode=/cluster2 Set watcher on existing znode=/cluster2/running 2023-10-15T16:04:48,927 DEBUG [pool-523-thread-1 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: INIT 2023-10-15T16:04:48,927 INFO [RegionServer-Restarter {}] hfile.BlockCacheFactory(127): Allocating BlockCache size=880 MB, blockSize=64 KB 2023-10-15T16:04:48,931 DEBUG [RegionServer-Restarter {}] mob.MobFileCache(121): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-10-15T16:04:48,931 DEBUG [RegionServer-Restarter {}] zookeeper.ZKUtil(111): regionserver:35419-0x101b6f97f510030, quorum=127.0.0.1:52753, baseZNode=/cluster2 Set watcher on existing znode=/cluster2/master 2023-10-15T16:04:48,933 DEBUG [RegionServer-Restarter {}] zookeeper.ZKUtil(113): regionserver:35419-0x101b6f97f510030, quorum=127.0.0.1:52753, baseZNode=/cluster2 Set watcher on znode that does not yet exist, /cluster2/acl 2023-10-15T16:04:48,933 DEBUG [RegionServer-Restarter {}] ipc.RpcExecutor(290): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35419 2023-10-15T16:04:48,938 DEBUG [RegionServer-Restarter {}] ipc.RpcExecutor(290): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35419 2023-10-15T16:04:48,938 DEBUG [RegionServer-Restarter {}] ipc.RpcExecutor(290): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35419 2023-10-15T16:04:48,939 DEBUG [RegionServer-Restarter {}] ipc.RpcExecutor(290): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35419 2023-10-15T16:04:48,939 DEBUG [RegionServer-Restarter {}] ipc.RpcExecutor(290): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35419 2023-10-15T16:04:48,941 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(730): ClusterId : b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:04:48,941 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-10-15T16:04:48,943 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-10-15T16:04:48,943 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-10-15T16:04:48,946 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-10-15T16:04:48,946 DEBUG [RS:3;8c5d72df62de:35419 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47b2eb1f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=8c5d72df62de/172.17.0.3:0 2023-10-15T16:04:48,961 DEBUG [RS:3;8c5d72df62de:35419 {}] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;8c5d72df62de:35419 2023-10-15T16:04:48,961 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.RegionServerCoprocessorHost(65): System coprocessor loading is enabled 2023-10-15T16:04:48,961 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.RegionServerCoprocessorHost(66): Table coprocessor loading is enabled 2023-10-15T16:04:48,961 DEBUG [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(816): About to register with Master. 2023-10-15T16:04:48,962 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(2608): reportForDuty to master=8c5d72df62de,33403,1697385859093 with port=35419, startcode=1697385888915 2023-10-15T16:04:48,962 DEBUG [RS:3;8c5d72df62de:35419 {}] ipc.RpcConnection(127): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-10-15T16:04:48,964 INFO [HMaster-EventLoopGroup-9-2 {}] ipc.ServerRpcConnection(478): Connection from 172.17.0.3:59811, version=4.0.0-alpha-1-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-10-15T16:04:48,965 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33403 {}] master.ServerManager(427): Registering regionserver=8c5d72df62de,35419,1697385888915 2023-10-15T16:04:48,966 DEBUG [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(1415): Config from master: hbase.rootdir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1 2023-10-15T16:04:48,967 DEBUG [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(1415): Config from master: fs.defaultFS=hdfs://localhost:39027 2023-10-15T16:04:48,967 DEBUG [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(1415): Config from master: hbase.master.info.port=-1 2023-10-15T16:04:48,969 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:38213-0x101b6f97f510005, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster2/rs 2023-10-15T16:04:48,969 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster2/rs 2023-10-15T16:04:48,969 DEBUG [RS:3;8c5d72df62de:35419 {}] zookeeper.ZKUtil(111): regionserver:35419-0x101b6f97f510030, quorum=127.0.0.1:52753, baseZNode=/cluster2 Set watcher on existing znode=/cluster2/rs/8c5d72df62de,35419,1697385888915 2023-10-15T16:04:48,969 WARN [RS:3;8c5d72df62de:35419 {}] hbase.ZNodeClearer(65): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-10-15T16:04:48,970 INFO [RS:3;8c5d72df62de:35419 {}] wal.WALFactory(196): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-10-15T16:04:48,970 INFO [RegionServerTracker-0 {}] master.RegionServerTracker(179): RegionServer ephemeral node created, adding [8c5d72df62de,35419,1697385888915] 2023-10-15T16:04:48,970 DEBUG [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(1748): logDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915 2023-10-15T16:04:48,996 INFO [RS:3;8c5d72df62de:35419 {}] hbase.HBaseConfiguration(255): Overriding client registry implementation to org.apache.hadoop.hbase.client.ZKConnectionRegistry 2023-10-15T16:04:48,997 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.MetricsRegionServerWrapperImpl(167): Computing regionserver metrics every 5000 milliseconds 2023-10-15T16:04:48,999 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.MemStoreFlusher(131): globalMemStoreLimit=880 M, globalMemStoreLimitLowMark=836 M, Offheap=false 2023-10-15T16:04:48,999 INFO [RS:3;8c5d72df62de:35419 {}] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-10-15T16:04:48,999 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(168): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-10-15T16:04:48,999 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer$CompactionChecker(1634): CompactionChecker runs every PT1S 2023-10-15T16:04:49,001 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ExecutorStatusChore(48): ExecutorStatusChore runs every 1mins, 0sec 2023-10-15T16:04:49,001 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(168): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_OPEN_REGION-regionserver/8c5d72df62de:0, corePoolSize=1, maxPoolSize=1 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_OPEN_META-regionserver/8c5d72df62de:0, corePoolSize=1, maxPoolSize=1 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0, corePoolSize=1, maxPoolSize=1 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_CLOSE_REGION-regionserver/8c5d72df62de:0, corePoolSize=1, maxPoolSize=1 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_CLOSE_META-regionserver/8c5d72df62de:0, corePoolSize=1, maxPoolSize=1 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/8c5d72df62de:0, corePoolSize=2, maxPoolSize=2 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/8c5d72df62de:0, corePoolSize=1, maxPoolSize=1 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/8c5d72df62de:0, corePoolSize=1, maxPoolSize=1 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_REFRESH_PEER-regionserver/8c5d72df62de:0, corePoolSize=1, maxPoolSize=1 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_REPLAY_SYNC_REPLICATION_WAL-regionserver/8c5d72df62de:0, corePoolSize=1, maxPoolSize=1 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/8c5d72df62de:0, corePoolSize=1, maxPoolSize=1 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0, corePoolSize=1, maxPoolSize=1 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_SNAPSHOT_OPERATIONS-regionserver/8c5d72df62de:0, corePoolSize=3, maxPoolSize=3 2023-10-15T16:04:49,001 DEBUG [RS:3;8c5d72df62de:35419 {}] executor.ExecutorService(95): Starting executor service name=RS_FLUSH_OPERATIONS-regionserver/8c5d72df62de:0, corePoolSize=3, maxPoolSize=3 2023-10-15T16:04:49,002 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(168): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-10-15T16:04:49,002 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(168): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-10-15T16:04:49,002 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(168): Chore ScheduledChore name=ExecutorStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-10-15T16:04:49,002 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(168): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-10-15T16:04:49,002 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(168): Chore ScheduledChore name=BrokenStoreFileCleaner, period=21600000, unit=MILLISECONDS is enabled. 2023-10-15T16:04:49,002 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(168): Chore ScheduledChore name=8c5d72df62de,35419,1697385888915-MobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-10-15T16:04:49,015 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-10-15T16:04:49,015 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(168): Chore ScheduledChore name=8c5d72df62de,35419,1697385888915-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-10-15T16:04:49,028 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,35419,1697385888915, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:49,028 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(168): Chore ScheduledChore name=ReplicationSourceStatistics, period=5000, unit=MILLISECONDS is enabled. 2023-10-15T16:04:49,028 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.Replication(171): 8c5d72df62de,35419,1697385888915 started 2023-10-15T16:04:49,042 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(168): Chore ScheduledChore name=ReplicationSinkStatistics, period=5000, unit=MILLISECONDS is enabled. 2023-10-15T16:04:49,042 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(1457): Serving as 8c5d72df62de,35419,1697385888915, RpcServer on 8c5d72df62de/172.17.0.3:35419, sessionid=0x101b6f97f510030 2023-10-15T16:04:49,042 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(86): Waiting RS [8c5d72df62de,35419,1697385888915] to online 2023-10-15T16:04:49,042 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(88): Waiting the old RS RS:0;8c5d72df62de:38435 thread to quit 2023-10-15T16:04:49,042 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(90): Done stop RS [8c5d72df62de,38435,1697385859175] and restart [8c5d72df62de,35419,1697385888915] 2023-10-15T16:04:49,042 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(80): Going to RS stop [8c5d72df62de,46527,1697385859222] 2023-10-15T16:04:49,042 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-10-15T16:04:49,042 INFO [RegionServer-Restarter {}] regionserver.HRegionServer(2147): ***** STOPPING region server '8c5d72df62de,46527,1697385859222' ***** 2023-10-15T16:04:49,042 INFO [RegionServer-Restarter {}] regionserver.HRegionServer(2161): STOPPED: Stop RS for test 2023-10-15T16:04:49,042 DEBUG [RS:3;8c5d72df62de:35419 {}] flush.RegionServerFlushTableProcedureManager(108): Start region server flush procedure manager 8c5d72df62de,35419,1697385888915 2023-10-15T16:04:49,042 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(131): Waiting for 8c5d72df62de,46527,1697385859222 to be listed as dead in master 2023-10-15T16:04:49,042 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.ZKProcedureMemberRpcs(357): Starting procedure member '8c5d72df62de,35419,1697385888915' 2023-10-15T16:04:49,042 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/cluster2/flush-table-proc/abort' 2023-10-15T16:04:49,042 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.HeapMemoryManager(220): Stopping 2023-10-15T16:04:49,042 INFO [MemStoreFlusher.0 {}] regionserver.MemStoreFlusher$FlushHandler(374): MemStoreFlusher.0 exiting 2023-10-15T16:04:49,043 INFO [RS:1;8c5d72df62de:46527 {}] flush.RegionServerFlushTableProcedureManager(119): Stopping region server flush procedure manager gracefully. 2023-10-15T16:04:49,043 INFO [RS:1;8c5d72df62de:46527 {}] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-10-15T16:04:49,043 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.HRegionServer(3035): Received CLOSE for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:49,043 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.HRegionServer(3035): Received CLOSE for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:49,043 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.HRegionServer(943): stopping server 8c5d72df62de,46527,1697385859222 2023-10-15T16:04:49,043 INFO [RS:1;8c5d72df62de:46527 {}] hbase.HBaseServerBase(452): Close async cluster connection 2023-10-15T16:04:49,043 INFO [RS:1;8c5d72df62de:46527 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:1;8c5d72df62de:46527. 2023-10-15T16:04:49,043 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1710): Closing 2d525cde81451d9cc8ed5e98268bcae5, disabling compactions & flushes 2023-10-15T16:04:49,044 DEBUG [RS:1;8c5d72df62de:46527 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.HBaseServerBase.closeClusterConnection(HBaseServerBase.java:454) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:946) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:49,044 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1743): Closing region hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:49,044 DEBUG [RS:1;8c5d72df62de:46527 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:49,044 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1764): Time limited wait for close lock on hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:49,044 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1831): Acquired close lock on hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. after waiting 0 ms 2023-10-15T16:04:49,044 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1841): Updates disabled for region hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:49,044 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.CompactSplit(469): Waiting for Split Thread to finish... 2023-10-15T16:04:49,044 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.CompactSplit(469): Waiting for Large Compaction Thread to finish... 2023-10-15T16:04:49,044 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.CompactSplit(469): Waiting for Small Compaction Thread to finish... 2023-10-15T16:04:49,044 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.HRegionServer(3035): Received CLOSE for 1588230740 2023-10-15T16:04:49,044 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(2885): Flushing 2d525cde81451d9cc8ed5e98268bcae5 3/3 column families, dataSize=1.11 KB heapSize=2.45 KB 2023-10-15T16:04:49,048 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.HRegionServer(1296): Waiting on 3 regions to close 2023-10-15T16:04:49,048 DEBUG [RS:1;8c5d72df62de:46527 {}] regionserver.HRegionServer(1300): Online Regions={1588230740=hbase:meta,,1.1588230740, 2d525cde81451d9cc8ed5e98268bcae5=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., f3bcdec0351888cb7677be57c1f453c1=SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1.} 2023-10-15T16:04:49,048 DEBUG [RS:1;8c5d72df62de:46527 {}] regionserver.HRegionServer(1326): Waiting on 1588230740, 2d525cde81451d9cc8ed5e98268bcae5, f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:49,049 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1710): Closing 1588230740, disabling compactions & flushes 2023-10-15T16:04:49,049 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1743): Closing region hbase:meta,,1.1588230740 2023-10-15T16:04:49,049 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/cluster2/flush-table-proc/acquired' 2023-10-15T16:04:49,049 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1764): Time limited wait for close lock on hbase:meta,,1.1588230740 2023-10-15T16:04:49,049 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1831): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-10-15T16:04:49,049 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1841): Updates disabled for region hbase:meta,,1.1588230740 2023-10-15T16:04:49,049 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(2885): Flushing 1588230740 4/4 column families, dataSize=1.89 KB heapSize=4.14 KB 2023-10-15T16:04:49,050 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-10-15T16:04:49,050 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-10-15T16:04:49,050 DEBUG [RS:3;8c5d72df62de:35419 {}] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager 8c5d72df62de,35419,1697385888915 2023-10-15T16:04:49,050 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.ZKProcedureMemberRpcs(357): Starting procedure member '8c5d72df62de,35419,1697385888915' 2023-10-15T16:04:49,050 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/cluster2/online-snapshot/abort' 2023-10-15T16:04:49,054 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/cluster2/online-snapshot/acquired' 2023-10-15T16:04:49,055 INFO [RS:3;8c5d72df62de:35419.replicationSource,1-8c5d72df62de,35419,1697385888915 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:49,056 DEBUG [RS:3;8c5d72df62de:35419 {}] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-10-15T16:04:49,056 INFO [RS:3;8c5d72df62de:35419 {}] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-10-15T16:04:49,056 INFO [RS:3;8c5d72df62de:35419 {}] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-10-15T16:04:49,065 DEBUG [RS:3;8c5d72df62de:35419.replicationSource,1-8c5d72df62de,35419,1697385888915-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:49,066 DEBUG [RS:3;8c5d72df62de:35419.replicationSource,1-8c5d72df62de,35419,1697385888915 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x5e0baefa to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:49,067 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/info/157091bd3617483a8caae084b2ec865f is 147, key is hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5./info:regioninfo/1697385881105/Put/seqid=0 2023-10-15T16:04:49,067 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/queue/d0ec0dbb35af4b308ac053911182fd96 is 193, key is 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285/queue:8c5d72df62de%2C45999%2C1697385859285.rep/1697385884956/Put/seqid=0 2023-10-15T16:04:49,089 INFO [regionserver/8c5d72df62de:0.leaseChecker {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:49,113 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f510031 connected 2023-10-15T16:04:49,113 DEBUG [RS:3;8c5d72df62de:35419.replicationSource,1-8c5d72df62de,35419,1697385888915 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@39f8702a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:49,114 INFO [RS:3;8c5d72df62de:35419.replicationSource,1-8c5d72df62de,35419,1697385888915 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,35419,1697385888915 (queues=0) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:49,127 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=1.11 KB at sequenceid=31 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/queue/d0ec0dbb35af4b308ac053911182fd96 2023-10-15T16:04:49,147 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d0ec0dbb35af4b308ac053911182fd96 2023-10-15T16:04:49,150 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/queue/d0ec0dbb35af4b308ac053911182fd96 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/d0ec0dbb35af4b308ac053911182fd96 2023-10-15T16:04:49,155 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=1.78 KB at sequenceid=24 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/info/157091bd3617483a8caae084b2ec865f 2023-10-15T16:04:49,157 INFO [RS:3;8c5d72df62de:35419 {}] monitor.StreamSlowMonitor(122): New stream slow monitor defaultMonitorName 2023-10-15T16:04:49,159 INFO [RS:3;8c5d72df62de:35419 {}] wal.AbstractFSWAL(602): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=8c5d72df62de%2C35419%2C1697385888915, suffix=, logDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915, archiveDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs, maxLogs=10 2023-10-15T16:04:49,159 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d0ec0dbb35af4b308ac053911182fd96 2023-10-15T16:04:49,160 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/d0ec0dbb35af4b308ac053911182fd96, entries=6, sequenceid=31, filesize=6.1 K 2023-10-15T16:04:49,163 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(3123): Finished flush of dataSize ~1.11 KB/1140, heapSize ~1.94 KB/1984, currentSize=0 B/0 for 2d525cde81451d9cc8ed5e98268bcae5 in 119ms, sequenceid=31, compaction requested=true 2023-10-15T16:04:49,182 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/rep_barrier/c98e652f22ac4a9e80d55c31b3cbcdf5 is 115, key is SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1./rep_barrier:seqnumDuringOpen/1697385881057/Put/seqid=0 2023-10-15T16:04:49,194 DEBUG [RS:3;8c5d72df62de:35419 {}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, exclude list is [], retry=0 2023-10-15T16:04:49,200 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:49,201 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:49,201 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] wal.WALSplitUtil(410): Wrote file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/recovered.edits/34.seqid, newMaxSeqId=34, maxSeqId=22 2023-10-15T16:04:49,201 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:49,202 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-15T16:04:49,204 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1961): Closed hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:49,204 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1664): Region close journal for 2d525cde81451d9cc8ed5e98268bcae5: Waiting for close lock at 1697385889043Running coprocessor pre-close hooks at 1697385889043Disabling compacts and flushes for region at 1697385889043Disabling writes for close at 1697385889044 (+1 ms)Obtaining lock to block concurrent updates at 1697385889044Preparing flush snapshotting stores in 2d525cde81451d9cc8ed5e98268bcae5 at 1697385889044Finished memstore snapshotting hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., syncing WAL and waiting on mvcc, flushsize=dataSize=1140, getHeapSize=2464, getOffHeapSize=0, getCellsCount=8 at 1697385889044Flushing stores of hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. at 1697385889045 (+1 ms)Flushing 2d525cde81451d9cc8ed5e98268bcae5/queue: creating writer at 1697385889045Flushing 2d525cde81451d9cc8ed5e98268bcae5/queue: appending metadata at 1697385889066 (+21 ms)Flushing 2d525cde81451d9cc8ed5e98268bcae5/queue: closing flushed file at 1697385889066Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@45d307be: reopening flushed file at 1697385889147 (+81 ms)Finished flush of dataSize ~1.11 KB/1140, heapSize ~1.94 KB/1984, currentSize=0 B/0 for 2d525cde81451d9cc8ed5e98268bcae5 in 119ms, sequenceid=31, compaction requested=true at 1697385889163 (+16 ms)Writing region close event to WAL at 1697385889194 (+31 ms)Running coprocessor post-close hooks at 1697385889202 (+8 ms)Closed at 1697385889204 (+2 ms) 2023-10-15T16:04:49,204 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] handler.CloseRegionHandler(117): Closed hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:49,212 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1710): Closing f3bcdec0351888cb7677be57c1f453c1, disabling compactions & flushes 2023-10-15T16:04:49,212 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1743): Closing region SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:49,212 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1764): Time limited wait for close lock on SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:49,212 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1831): Acquired close lock on SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. after waiting 0 ms 2023-10-15T16:04:49,212 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1841): Updates disabled for region SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:49,215 INFO [RS:3;8c5d72df62de:35419 {}] wal.AbstractFSWAL(980): New WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 2023-10-15T16:04:49,215 DEBUG [RS:3;8c5d72df62de:35419 {}] regionserver.ReplicationSourceManager(789): Start tracking logs for wal group 8c5d72df62de%2C35419%2C1697385888915 for peer 1 2023-10-15T16:04:49,215 DEBUG [RS:3;8c5d72df62de:35419 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C35419%2C1697385888915 2023-10-15T16:04:49,216 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,35419,1697385888915, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:49,219 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=111 B at sequenceid=24 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/rep_barrier/c98e652f22ac4a9e80d55c31b3cbcdf5 2023-10-15T16:04:49,224 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] wal.WALSplitUtil(410): Wrote file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-10-15T16:04:49,225 DEBUG [RS:3;8c5d72df62de:35419 {}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK], DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK], DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK]] 2023-10-15T16:04:49,225 INFO [RS:3;8c5d72df62de:35419.replicationSource.shipper8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C35419%2C1697385888915 2023-10-15T16:04:49,225 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1961): Closed SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:49,225 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1664): Region close journal for f3bcdec0351888cb7677be57c1f453c1: Waiting for close lock at 1697385889205Running coprocessor pre-close hooks at 1697385889205Disabling compacts and flushes for region at 1697385889205Disabling writes for close at 1697385889212 (+7 ms)Writing region close event to WAL at 1697385889213 (+1 ms)Running coprocessor post-close hooks at 1697385889225 (+12 ms)Closed at 1697385889225 2023-10-15T16:04:49,228 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] handler.CloseRegionHandler(117): Closed SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:49,228 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=0, beingWritten=true 2023-10-15T16:04:49,232 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/info/157091bd3617483a8caae084b2ec865f as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/info/157091bd3617483a8caae084b2ec865f 2023-10-15T16:04:49,238 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/info/157091bd3617483a8caae084b2ec865f, entries=16, sequenceid=24, filesize=7.1 K 2023-10-15T16:04:49,240 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/.tmp/rep_barrier/c98e652f22ac4a9e80d55c31b3cbcdf5 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/rep_barrier/c98e652f22ac4a9e80d55c31b3cbcdf5 2023-10-15T16:04:49,248 DEBUG [RS:1;8c5d72df62de:46527 {}] regionserver.HRegionServer(1326): Waiting on 1588230740 2023-10-15T16:04:49,248 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/rep_barrier/c98e652f22ac4a9e80d55c31b3cbcdf5, entries=1, sequenceid=24, filesize=5.4 K 2023-10-15T16:04:49,250 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(3123): Finished flush of dataSize ~1.89 KB/1931, heapSize ~3.61 KB/3696, currentSize=0 B/0 for 1588230740 in 201ms, sequenceid=24, compaction requested=false 2023-10-15T16:04:49,263 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] wal.WALSplitUtil(410): Wrote file=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/recovered.edits/27.seqid, newMaxSeqId=27, maxSeqId=17 2023-10-15T16:04:49,264 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-15T16:04:49,265 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1961): Closed hbase:meta,,1.1588230740 2023-10-15T16:04:49,265 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1664): Region close journal for 1588230740: Waiting for close lock at 1697385889048Running coprocessor pre-close hooks at 1697385889048Disabling compacts and flushes for region at 1697385889048Disabling writes for close at 1697385889049 (+1 ms)Obtaining lock to block concurrent updates at 1697385889049Preparing flush snapshotting stores in 1588230740 at 1697385889049Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=1931, getHeapSize=4176, getOffHeapSize=0, getCellsCount=17 at 1697385889050 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1697385889050Flushing 1588230740/info: creating writer at 1697385889054 (+4 ms)Flushing 1588230740/info: appending metadata at 1697385889067 (+13 ms)Flushing 1588230740/info: closing flushed file at 1697385889067Flushing 1588230740/rep_barrier: creating writer at 1697385889161 (+94 ms)Flushing 1588230740/rep_barrier: appending metadata at 1697385889181 (+20 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1697385889181Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@60c9985f: reopening flushed file at 1697385889224 (+43 ms)Flushing org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl@3d60e17c: reopening flushed file at 1697385889238 (+14 ms)Finished flush of dataSize ~1.89 KB/1931, heapSize ~3.61 KB/3696, currentSize=0 B/0 for 1588230740 in 201ms, sequenceid=24, compaction requested=false at 1697385889250 (+12 ms)Writing region close event to WAL at 1697385889259 (+9 ms)Running coprocessor post-close hooks at 1697385889264 (+5 ms)Closed at 1697385889265 (+1 ms) 2023-10-15T16:04:49,265 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-10-15T16:04:49,437 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 0, reset compression=false 2023-10-15T16:04:49,449 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.HRegionServer(960): stopping server 8c5d72df62de,46527,1697385859222; all regions closed. 2023-10-15T16:04:49,456 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] replication.HBaseReplicationEndpoint(252): No sinks available at peer. Will not be able to replicate 2023-10-15T16:04:49,470 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=0, beingWritten=true 2023-10-15T16:04:49,474 DEBUG [RS:1;8c5d72df62de:46527 {}] wal.AbstractFSWAL(1244): Moved 1 WAL file(s) to /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs 2023-10-15T16:04:49,474 INFO [RS:1;8c5d72df62de:46527 {}] wal.AbstractFSWAL(1247): Closed WAL: AsyncFSWAL 8c5d72df62de%2C46527%2C1697385859222.meta:.meta(num 1697385880058) 2023-10-15T16:04:49,481 DEBUG [RS:1;8c5d72df62de:46527 {}] wal.AbstractFSWAL(1244): Moved 1 WAL file(s) to /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs 2023-10-15T16:04:49,481 INFO [RS:1;8c5d72df62de:46527 {}] wal.AbstractFSWAL(1247): Closed WAL: AsyncFSWAL 8c5d72df62de%2C46527%2C1697385859222.rep:(num 1697385880933) 2023-10-15T16:04:49,494 DEBUG [RS:1;8c5d72df62de:46527 {}] wal.AbstractFSWAL(1244): Moved 1 WAL file(s) to /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs 2023-10-15T16:04:49,494 INFO [RS:1;8c5d72df62de:46527 {}] wal.AbstractFSWAL(1247): Closed WAL: AsyncFSWAL 8c5d72df62de%2C46527%2C1697385859222:(num 1697385867323) 2023-10-15T16:04:49,494 DEBUG [RS:1;8c5d72df62de:46527 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:49,495 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:49,495 INFO [RS:1;8c5d72df62de:46527 {}] hbase.HBaseServerBase(435): Shutdown chores and chore service 2023-10-15T16:04:49,495 INFO [RS:1;8c5d72df62de:46527 {}] hbase.ChoreService(370): Chore service for: regionserver/8c5d72df62de:0 had [ScheduledChore name=ReplicationSinkStatistics, period=5000, unit=MILLISECONDS, ScheduledChore name=ReplicationSourceStatistics, period=5000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-10-15T16:04:49,495 INFO [RS:1;8c5d72df62de:46527 {}] hbase.HBaseServerBase(445): Shutdown executor service 2023-10-15T16:04:49,495 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(249): LogRoller exiting. 2023-10-15T16:04:49,495 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,46527,1697385859222 because: Region server is closing 2023-10-15T16:04:49,600 DEBUG [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f510020, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:49,601 INFO [RS:1;8c5d72df62de:46527 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:1;8c5d72df62de:46527. 2023-10-15T16:04:49,601 DEBUG [RS:1;8c5d72df62de:46527 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:664) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:659) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:987) at org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:160) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2481) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:995) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:49,601 DEBUG [RS:1;8c5d72df62de:46527 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:49,601 DEBUG [RS:1;8c5d72df62de:46527 {}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x2106402e to 127.0.0.1:52753 2023-10-15T16:04:49,702 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] util.Threads(127): sleep interrupted java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) ~[?:?] at org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:125) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.sleep(ReplicationSourceWALReader.java:128) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:160) ~[classes/:?] 2023-10-15T16:04:49,704 WARN [RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) ~[?:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:307) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) ~[classes/:?] 2023-10-15T16:04:49,704 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_REFRESH_PEER-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,46527,1697385859222 terminated 2023-10-15T16:04:49,704 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 because: Region server is closing 2023-10-15T16:04:49,779 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 0, reset compression=false 2023-10-15T16:04:49,784 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=0, beingWritten=true 2023-10-15T16:04:49,807 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f51002c, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:49,807 INFO [RS:1;8c5d72df62de:46527 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:1;8c5d72df62de:46527. 2023-10-15T16:04:49,807 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f51002c, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:49,807 DEBUG [RS:1;8c5d72df62de:46527 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:664) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:659) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:991) at org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:160) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2481) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:995) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:49,808 DEBUG [RS:1;8c5d72df62de:46527 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:49,808 DEBUG [RS:1;8c5d72df62de:46527 {}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x57db6dea to 127.0.0.1:52753 2023-10-15T16:04:49,909 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.HBaseInterClusterReplicationEndpoint(181): Waiting for peers [Source for peer 1]: Interrupted while sleeping between retries 2023-10-15T16:04:49,909 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.HBaseInterClusterReplicationEndpoint(442): No replication sinks found, returning without replicating. The source should retry with the same set of edits. Not logging this again for the next 10 seconds. 2023-10-15T16:04:49,909 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 {}] regionserver.HBaseInterClusterReplicationEndpoint(181): No sinks available at peer [Source for peer 1]: Interrupted while sleeping between retries 2023-10-15T16:04:49,909 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285.replicationSource.shipper8c5d72df62de%2C45999%2C1697385859285,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 terminated 2023-10-15T16:04:49,909 INFO [RS:1;8c5d72df62de:46527 {}] ipc.NettyRpcServer(340): Stopping server on /172.17.0.3:46527 2023-10-15T16:04:49,917 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster2/rs 2023-10-15T16:04:49,917 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:38213-0x101b6f97f510005, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster2/rs 2023-10-15T16:04:49,918 INFO [RS:1;8c5d72df62de:46527 {}] hbase.HBaseServerBase(476): Close zookeeper 2023-10-15T16:04:49,918 INFO [RegionServerTracker-0 {}] master.RegionServerTracker(171): RegionServer ephemeral node deleted, processing expiration [8c5d72df62de,46527,1697385859222] 2023-10-15T16:04:49,921 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): regionserver:46527-0x101b6f97f510008, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster2/rs/8c5d72df62de,46527,1697385859222 2023-10-15T16:04:49,922 DEBUG [RegionServerTracker-0 {}] zookeeper.RecoverableZooKeeper(221): Node /cluster2/draining/8c5d72df62de,46527,1697385859222 already deleted, retry=false 2023-10-15T16:04:49,922 INFO [RegionServerTracker-0 {}] master.ServerManager(604): Processing expiration of 8c5d72df62de,46527,1697385859222 on 8c5d72df62de,33403,1697385859093 2023-10-15T16:04:49,930 DEBUG [RegionServerTracker-0 {}] procedure2.ProcedureExecutor(1032): Stored pid=56, state=RUNNABLE:SERVER_CRASH_START, hasLock=false; ServerCrashProcedure 8c5d72df62de,46527,1697385859222, splitWal=true, meta=true 2023-10-15T16:04:49,930 INFO [RegionServerTracker-0 {}] assignment.AssignmentManager(1862): Scheduled ServerCrashProcedure pid=56 for 8c5d72df62de,46527,1697385859222 (carryingMeta=true) 8c5d72df62de,46527,1697385859222/CRASHED/regionCount=3/lock=java.util.concurrent.locks.ReentrantReadWriteLock@42a8dcc7[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-10-15T16:04:49,932 INFO [master/8c5d72df62de:0.Chore.1 {}] master.ServerManager(1043): Rewriting .lastflushedseqids file at: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/.lastflushedseqids 2023-10-15T16:04:49,934 INFO [PEWorker-2 {}] procedure.ServerCrashProcedure(163): Start pid=56, state=RUNNABLE:SERVER_CRASH_START, hasLock=true; ServerCrashProcedure 8c5d72df62de,46527,1697385859222, splitWal=true, meta=true 2023-10-15T16:04:49,937 INFO [PEWorker-2 {}] procedure.ServerCrashProcedure(333): Splitting WALs pid=56, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, hasLock=true; ServerCrashProcedure 8c5d72df62de,46527,1697385859222, splitWal=true, meta=true, isMeta: true 2023-10-15T16:04:49,938 DEBUG [PEWorker-2 {}] master.MasterWalManager(317): Renamed region directory: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222-splitting 2023-10-15T16:04:49,940 INFO [PEWorker-2 {}] master.SplitLogManager(171): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222-splitting dir is empty, no logs to split. 2023-10-15T16:04:49,940 INFO [PEWorker-2 {}] master.SplitWALManager(105): 8c5d72df62de,46527,1697385859222 WAL count=0, meta=true 2023-10-15T16:04:49,943 INFO [PEWorker-2 {}] master.SplitLogManager(171): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222-splitting dir is empty, no logs to split. 2023-10-15T16:04:49,943 INFO [PEWorker-2 {}] master.SplitWALManager(105): 8c5d72df62de,46527,1697385859222 WAL count=0, meta=true 2023-10-15T16:04:49,943 DEBUG [PEWorker-2 {}] procedure.ServerCrashProcedure(323): Check if 8c5d72df62de,46527,1697385859222 WAL splitting is done? wals=0, meta=true 2023-10-15T16:04:49,944 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-10-15T16:04:49,946 INFO [PEWorker-2 {}] procedure.MasterProcedureScheduler(792): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-10-15T16:04:49,947 INFO [PEWorker-2 {}] assignment.TransitRegionStateProcedure(262): Starting pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-10-15T16:04:50,031 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): regionserver:46527-0x101b6f97f510008, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:50,031 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): regionserver:46527-0x101b6f97f510008, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:50,031 INFO [RS:1;8c5d72df62de:46527 {}] hbase.HBaseServerBase(483): Close table descriptors 2023-10-15T16:04:50,031 INFO [RS:1;8c5d72df62de:46527 {}] regionserver.HRegionServer(1015): Exiting; stopping=8c5d72df62de,46527,1697385859222; zookeeper connection closed. 2023-10-15T16:04:50,032 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@68cbed20 {}] hbase.SingleProcessHBaseCluster$SingleFileSystemShutdownThread(211): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@68cbed20 2023-10-15T16:04:50,042 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(134): Server 8c5d72df62de,46527,1697385859222 marked as dead, waiting for it to finish dead processing 2023-10-15T16:04:50,042 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:04:50,097 DEBUG [8c5d72df62de:33403 {}] assignment.AssignmentManager(2316): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-10-15T16:04:50,101 INFO [PEWorker-3 {}] assignment.RegionStateStore(223): pid=57 updating hbase:meta row=1588230740, regionState=OPENING, regionLocation=8c5d72df62de,35419,1697385888915 2023-10-15T16:04:50,103 INFO [PEWorker-3 {}] zookeeper.MetaTableLocator(171): Setting hbase:meta replicaId=0 location in ZooKeeper as 8c5d72df62de,35419,1697385888915, state=OPENING 2023-10-15T16:04:50,104 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:38213-0x101b6f97f510005, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:50,104 DEBUG [RegionServer-Restarter-EventThread {}] zookeeper.ZKWatcher(604): regionserver:35419-0x101b6f97f510030, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:50,104 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:50,104 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:50,104 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE, hasLock=false; OpenRegionProcedure 1588230740, server=8c5d72df62de,35419,1697385888915}] 2023-10-15T16:04:50,104 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:50,105 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:50,188 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 0, reset compression=false 2023-10-15T16:04:50,198 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=0, beingWritten=true 2023-10-15T16:04:50,262 DEBUG [RSProcedureDispatcher-pool-0 {}] ipc.RpcConnection(127): Using SIMPLE authentication for service=AdminService, sasl=false 2023-10-15T16:04:50,264 INFO [MiniHBaseClusterRegionServer-EventLoopGroup-13-3 {}] ipc.ServerRpcConnection(478): Connection from 172.17.0.3:46245, version=4.0.0-alpha-1-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-10-15T16:04:50,268 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] handler.AssignRegionHandler(127): Open hbase:meta,,1.1588230740 2023-10-15T16:04:50,268 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] wal.WALFactory(196): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-10-15T16:04:50,268 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] monitor.StreamSlowMonitor(122): New stream slow monitor .meta 2023-10-15T16:04:50,273 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] wal.AbstractFSWAL(602): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=8c5d72df62de%2C35419%2C1697385888915.meta, suffix=.meta, logDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915, archiveDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs, maxLogs=10 2023-10-15T16:04:50,288 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.meta.1697385890273.meta, exclude list is [], retry=0 2023-10-15T16:04:50,291 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:50,291 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:50,291 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:50,300 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] wal.AbstractFSWAL(980): New WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.meta.1697385890273.meta 2023-10-15T16:04:50,300 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK], DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK], DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK]] 2023-10-15T16:04:50,301 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(7727): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-10-15T16:04:50,301 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-10-15T16:04:50,301 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(8255): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-10-15T16:04:50,301 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.RegionCoprocessorHost(391): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-10-15T16:04:50,301 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-10-15T16:04:50,301 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(891): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=0 ; minColumnNum=100 ; preparePutThreadLimit=0 ; hotProtect now disable 2023-10-15T16:04:50,301 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(7769): checking encryption for 1588230740 2023-10-15T16:04:50,301 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(7772): checking classloading for 1588230740 2023-10-15T16:04:50,303 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-10-15T16:04:50,304 INFO [StoreOpener-1588230740-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-10-15T16:04:50,304 DEBUG [StoreOpener-1588230740-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:50,310 DEBUG [StoreOpener-1588230740-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/info/157091bd3617483a8caae084b2ec865f 2023-10-15T16:04:50,315 DEBUG [StoreOpener-1588230740-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/info/846f92233df94355b7178307130e3f39 2023-10-15T16:04:50,316 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(324): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=ROW_INDEX_V1, compression=NONE 2023-10-15T16:04:50,316 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family ns of region 1588230740 2023-10-15T16:04:50,316 INFO [StoreOpener-1588230740-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName ns 2023-10-15T16:04:50,317 DEBUG [StoreOpener-1588230740-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:50,325 DEBUG [StoreOpener-1588230740-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/ns/4551d862e332470c8a8a259837c35f56 2023-10-15T16:04:50,325 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(324): Store=1588230740/ns, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=ROW_INDEX_V1, compression=NONE 2023-10-15T16:04:50,325 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-10-15T16:04:50,326 INFO [StoreOpener-1588230740-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-10-15T16:04:50,326 DEBUG [StoreOpener-1588230740-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:50,332 DEBUG [StoreOpener-1588230740-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/rep_barrier/1243340783024e4996a370ac9696d649 2023-10-15T16:04:50,339 DEBUG [StoreOpener-1588230740-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/rep_barrier/c98e652f22ac4a9e80d55c31b3cbcdf5 2023-10-15T16:04:50,339 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(324): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=ROW_INDEX_V1, compression=NONE 2023-10-15T16:04:50,339 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-10-15T16:04:50,340 INFO [StoreOpener-1588230740-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-10-15T16:04:50,340 DEBUG [StoreOpener-1588230740-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:50,350 DEBUG [StoreOpener-1588230740-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740/table/e4e23e8fab1543999e04dda4f557f657 2023-10-15T16:04:50,353 INFO [StoreOpener-1588230740-1 {}] regionserver.HStore(324): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=ROW_INDEX_V1, compression=NONE 2023-10-15T16:04:50,354 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(1028): replaying wal for 1588230740 2023-10-15T16:04:50,355 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740 2023-10-15T16:04:50,356 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/meta/1588230740 2023-10-15T16:04:50,358 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(1038): stopping wal replay for 1588230740 2023-10-15T16:04:50,358 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(1050): Cleaning up temporary data for 1588230740 2023-10-15T16:04:50,359 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (32.0 M)) instead. 2023-10-15T16:04:50,361 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(1083): writing seq id for 1588230740 2023-10-15T16:04:50,363 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(1104): Opened 1588230740; next sequenceid=28; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68652770, jitterRate=0.02300599217414856}}}, FlushLargeStoresPolicy{flushSizeLowerBound=33554432} 2023-10-15T16:04:50,363 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(1112): Running coprocessor post-open hooks for 1588230740 2023-10-15T16:04:50,364 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegion(996): Region open journal for 1588230740: Running coprocessor pre-open hook at 1697385890302Writing region info on filesystem at 1697385890302Initializing all the Stores at 1697385890303 (+1 ms)Instantiating store for column family {NAME => 'info', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'ROW_INDEX_V1', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROWCOL', IN_MEMORY => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '8192 B (8KB)'} at 1697385890303Instantiating store for column family {NAME => 'ns', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'ROW_INDEX_V1', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROWCOL', IN_MEMORY => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '8192 B (8KB)'} at 1697385890303Instantiating store for column family {NAME => 'rep_barrier', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'ROW_INDEX_V1', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROWCOL', IN_MEMORY => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385890303Instantiating store for column family {NAME => 'table', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'ROW_INDEX_V1', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROWCOL', IN_MEMORY => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '8192 B (8KB)'} at 1697385890303Cleaning up temporary data from old regions at 1697385890358 (+55 ms)Running coprocessor post-open hooks at 1697385890363 (+5 ms)Region opened successfully at 1697385890364 (+1 ms) 2023-10-15T16:04:50,365 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegionServer(2186): Post open deploy tasks for hbase:meta,,1.1588230740, pid=58, masterSystemTime=1697385890262 2023-10-15T16:04:50,368 DEBUG [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] regionserver.HRegionServer(2216): Finished post open deploy task for hbase:meta,,1.1588230740 2023-10-15T16:04:50,368 INFO [RS_OPEN_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_META, pid=58}] handler.AssignRegionHandler(147): Opened hbase:meta,,1.1588230740 2023-10-15T16:04:50,369 INFO [PEWorker-1 {}] assignment.RegionStateStore(223): pid=57 updating hbase:meta row=1588230740, regionState=OPEN, openSeqNum=28, regionLocation=8c5d72df62de,35419,1697385888915 2023-10-15T16:04:50,370 INFO [PEWorker-1 {}] zookeeper.MetaTableLocator(171): Setting hbase:meta replicaId=0 location in ZooKeeper as 8c5d72df62de,35419,1697385888915, state=OPEN 2023-10-15T16:04:50,371 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:38213-0x101b6f97f510005, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:50,371 DEBUG [RegionServer-Restarter-EventThread {}] zookeeper.ZKWatcher(604): regionserver:35419-0x101b6f97f510030, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:50,371 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/cluster2/meta-region-server 2023-10-15T16:04:50,371 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:50,371 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:50,371 DEBUG [zk-event-processor-pool-0 {}] hbase.MetaRegionLocationCache(167): Updating meta znode for path /cluster2/meta-region-server: CHANGED 2023-10-15T16:04:50,374 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=58, resume processing ppid=57 2023-10-15T16:04:50,375 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=58, ppid=57, state=SUCCESS, hasLock=false; OpenRegionProcedure 1588230740, server=8c5d72df62de,35419,1697385888915 in 268 msec 2023-10-15T16:04:50,377 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=57, resume processing ppid=56 2023-10-15T16:04:50,377 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1413): Finished pid=57, ppid=56, state=SUCCESS, hasLock=false; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 431 msec 2023-10-15T16:04:50,377 INFO [PEWorker-2 {}] procedure.ServerCrashProcedure(201): 8c5d72df62de,46527,1697385859222 had 3 regions 2023-10-15T16:04:50,378 INFO [PEWorker-2 {}] procedure.ServerCrashProcedure(333): Splitting WALs pid=56, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, hasLock=true; ServerCrashProcedure 8c5d72df62de,46527,1697385859222, splitWal=true, meta=true, isMeta: false 2023-10-15T16:04:50,381 INFO [PEWorker-2 {}] master.SplitLogManager(171): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222-splitting dir is empty, no logs to split. 2023-10-15T16:04:50,381 INFO [PEWorker-2 {}] master.SplitWALManager(105): 8c5d72df62de,46527,1697385859222 WAL count=0, meta=false 2023-10-15T16:04:50,383 INFO [PEWorker-2 {}] master.SplitLogManager(171): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,46527,1697385859222-splitting dir is empty, no logs to split. 2023-10-15T16:04:50,383 INFO [PEWorker-2 {}] master.SplitWALManager(105): 8c5d72df62de,46527,1697385859222 WAL count=0, meta=false 2023-10-15T16:04:50,383 DEBUG [PEWorker-2 {}] procedure.ServerCrashProcedure(323): Check if 8c5d72df62de,46527,1697385859222 WAL splitting is done? wals=0, meta=false 2023-10-15T16:04:50,387 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=SyncRep, region=f3bcdec0351888cb7677be57c1f453c1, ASSIGN}] 2023-10-15T16:04:50,388 INFO [PEWorker-4 {}] procedure.MasterProcedureScheduler(792): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN 2023-10-15T16:04:50,388 INFO [PEWorker-3 {}] procedure.MasterProcedureScheduler(792): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=false; TransitRegionStateProcedure table=SyncRep, region=f3bcdec0351888cb7677be57c1f453c1, ASSIGN 2023-10-15T16:04:50,390 INFO [PEWorker-4 {}] assignment.TransitRegionStateProcedure(262): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-10-15T16:04:50,390 INFO [PEWorker-3 {}] assignment.TransitRegionStateProcedure(262): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, hasLock=true; TransitRegionStateProcedure table=SyncRep, region=f3bcdec0351888cb7677be57c1f453c1, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-10-15T16:04:50,541 DEBUG [8c5d72df62de:33403 {}] assignment.AssignmentManager(2316): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-10-15T16:04:50,541 INFO [PEWorker-1 {}] assignment.RegionStateStore(223): pid=59 updating hbase:meta row=2d525cde81451d9cc8ed5e98268bcae5, regionState=OPENING, regionLocation=8c5d72df62de,35419,1697385888915 2023-10-15T16:04:50,541 INFO [PEWorker-5 {}] assignment.RegionStateStore(223): pid=60 updating hbase:meta row=f3bcdec0351888cb7677be57c1f453c1, regionState=OPENING, regionLocation=8c5d72df62de,35419,1697385888915 2023-10-15T16:04:50,542 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:46527 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:46527 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:50,543 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,46527,1697385859222, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,46527,1697385859222, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:46527 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:46527 2023-10-15T16:04:50,543 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,46527,1697385859222, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:04:50,543 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,46527,1697385859222, seqNum=-1 from cache 2023-10-15T16:04:50,543 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,46527,1697385859222, seqNum=-1 , the old value is null, error=java.net.ConnectException: Call to address=8c5d72df62de:46527 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:46527 2023-10-15T16:04:50,543 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:46527 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:46527 2023-10-15T16:04:50,650 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:04:50,650 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:04:50,650 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.RpcConnection(127): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-15T16:04:50,652 INFO [MiniHBaseClusterRegionServer-EventLoopGroup-13-1 {}] ipc.ServerRpcConnection(478): Connection from 172.17.0.3:50953, version=4.0.0-alpha-1-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-10-15T16:04:50,658 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE, hasLock=false; OpenRegionProcedure 2d525cde81451d9cc8ed5e98268bcae5, server=8c5d72df62de,35419,1697385888915}] 2023-10-15T16:04:50,661 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE, hasLock=false; OpenRegionProcedure f3bcdec0351888cb7677be57c1f453c1, server=8c5d72df62de,35419,1697385888915}] 2023-10-15T16:04:50,705 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 0, reset compression=false 2023-10-15T16:04:50,712 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=0, beingWritten=true 2023-10-15T16:04:50,816 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] handler.AssignRegionHandler(127): Open hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:50,816 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] wal.WALFactory(196): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-10-15T16:04:50,817 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] monitor.StreamSlowMonitor(122): New stream slow monitor rep 2023-10-15T16:04:50,818 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] wal.AbstractFSWAL(602): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=8c5d72df62de%2C35419%2C1697385888915.rep, suffix=, logDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915, archiveDir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs, maxLogs=10 2023-10-15T16:04:50,819 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] handler.AssignRegionHandler(127): Open SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:50,819 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(7727): Opening region: {ENCODED => f3bcdec0351888cb7677be57c1f453c1, NAME => 'SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1.', STARTKEY => '', ENDKEY => ''} 2023-10-15T16:04:50,820 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table SyncRep f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:50,820 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(891): Instantiated SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=0 ; minColumnNum=100 ; preparePutThreadLimit=0 ; hotProtect now disable 2023-10-15T16:04:50,820 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(7769): checking encryption for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:50,820 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(7772): checking classloading for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:50,824 INFO [StoreOpener-f3bcdec0351888cb7677be57c1f453c1-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:50,828 INFO [StoreOpener-f3bcdec0351888cb7677be57c1f453c1-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f3bcdec0351888cb7677be57c1f453c1 columnFamilyName cf 2023-10-15T16:04:50,828 DEBUG [StoreOpener-f3bcdec0351888cb7677be57c1f453c1-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:50,836 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.rep.1697385890819, exclude list is [], retry=0 2023-10-15T16:04:50,842 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:04:50,842 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:04:50,842 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:04:50,845 DEBUG [StoreOpener-f3bcdec0351888cb7677be57c1f453c1-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1/cf/9e5ff1a6ebb340fe803f1a85f55dc61b 2023-10-15T16:04:50,846 INFO [StoreOpener-f3bcdec0351888cb7677be57c1f453c1-1 {}] regionserver.HStore(324): Store=f3bcdec0351888cb7677be57c1f453c1/cf, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:50,846 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(1028): replaying wal for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:50,847 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:50,850 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/default/SyncRep/f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:50,850 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(1038): stopping wal replay for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:50,851 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(1050): Cleaning up temporary data for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:50,853 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(1083): writing seq id for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:50,854 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(1104): Opened f3bcdec0351888cb7677be57c1f453c1; next sequenceid=8; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62504610, jitterRate=-0.06860873103141785}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-10-15T16:04:50,854 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(1112): Running coprocessor post-open hooks for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:50,855 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegion(996): Region open journal for f3bcdec0351888cb7677be57c1f453c1: Running coprocessor pre-open hook at 1697385890820Writing region info on filesystem at 1697385890820Initializing all the Stores at 1697385890821 (+1 ms)Instantiating store for column family {NAME => 'cf', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385890823 (+2 ms)Cleaning up temporary data from old regions at 1697385890851 (+28 ms)Running coprocessor post-open hooks at 1697385890854 (+3 ms)Region opened successfully at 1697385890855 (+1 ms) 2023-10-15T16:04:50,856 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegionServer(2186): Post open deploy tasks for SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1., pid=62, masterSystemTime=1697385890812 2023-10-15T16:04:50,860 DEBUG [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] regionserver.HRegionServer(2216): Finished post open deploy task for SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:50,860 INFO [RS_OPEN_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_REGION, pid=62}] handler.AssignRegionHandler(147): Opened SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:50,860 INFO [PEWorker-3 {}] assignment.RegionStateStore(223): pid=60 updating hbase:meta row=f3bcdec0351888cb7677be57c1f453c1, regionState=OPEN, repBarrier=8, openSeqNum=8, regionLocation=8c5d72df62de,35419,1697385888915 2023-10-15T16:04:50,866 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] wal.AbstractFSWAL(980): New WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.rep.1697385890819 2023-10-15T16:04:50,866 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=62, resume processing ppid=60 2023-10-15T16:04:50,866 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1413): Finished pid=62, ppid=60, state=SUCCESS, hasLock=false; OpenRegionProcedure f3bcdec0351888cb7677be57c1f453c1, server=8c5d72df62de,35419,1697385888915 in 201 msec 2023-10-15T16:04:50,868 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=60, ppid=56, state=SUCCESS, hasLock=false; TransitRegionStateProcedure table=SyncRep, region=f3bcdec0351888cb7677be57c1f453c1, ASSIGN in 479 msec 2023-10-15T16:04:50,872 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK], DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK], DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK]] 2023-10-15T16:04:50,872 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(7727): Opening region: {ENCODED => 2d525cde81451d9cc8ed5e98268bcae5, NAME => 'hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.', STARTKEY => '', ENDKEY => ''} 2023-10-15T16:04:50,872 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-10-15T16:04:50,872 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(8255): Registered coprocessor service: region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. service=MultiRowMutationService 2023-10-15T16:04:50,872 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.RegionCoprocessorHost(391): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:replication successfully. 2023-10-15T16:04:50,872 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table replication 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,872 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(891): Instantiated hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=0 ; minColumnNum=100 ; preparePutThreadLimit=0 ; hotProtect now disable 2023-10-15T16:04:50,872 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(7769): checking encryption for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,872 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(7772): checking classloading for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,880 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family hfileref of region 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,881 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d525cde81451d9cc8ed5e98268bcae5 columnFamilyName hfileref 2023-10-15T16:04:50,884 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:50,891 INFO [StoreFileOpener-2d525cde81451d9cc8ed5e98268bcae5-hfileref-1 {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 06c1db2ad5514d38945c79bef8c56cc7 2023-10-15T16:04:50,891 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/hfileref/06c1db2ad5514d38945c79bef8c56cc7 2023-10-15T16:04:50,891 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(324): Store=2d525cde81451d9cc8ed5e98268bcae5/hfileref, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:50,891 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family queue of region 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,892 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d525cde81451d9cc8ed5e98268bcae5 columnFamilyName queue 2023-10-15T16:04:50,892 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:50,914 INFO [StoreFileOpener-2d525cde81451d9cc8ed5e98268bcae5-queue-1 {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 46fdfd784a1242dd9339a7a4fc97d13a 2023-10-15T16:04:50,914 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/46fdfd784a1242dd9339a7a4fc97d13a 2023-10-15T16:04:50,926 INFO [StoreFileOpener-2d525cde81451d9cc8ed5e98268bcae5-queue-1 {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8807426219fd436c817c02ff77937323 2023-10-15T16:04:50,926 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/8807426219fd436c817c02ff77937323 2023-10-15T16:04:50,943 INFO [StoreFileOpener-2d525cde81451d9cc8ed5e98268bcae5-queue-1 {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d0ec0dbb35af4b308ac053911182fd96 2023-10-15T16:04:50,943 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/d0ec0dbb35af4b308ac053911182fd96 2023-10-15T16:04:50,943 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(324): Store=2d525cde81451d9cc8ed5e98268bcae5/queue, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:50,943 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(397): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family sid of region 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,944 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] compactions.CompactionConfiguration(175): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d525cde81451d9cc8ed5e98268bcae5 columnFamilyName sid 2023-10-15T16:04:50,944 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] storefiletracker.StoreFileTrackerFactory(122): instantiating StoreFileTracker impl org.apache.hadoop.hbase.regionserver.storefiletracker.DefaultStoreFileTracker 2023-10-15T16:04:50,953 INFO [StoreFileOpener-2d525cde81451d9cc8ed5e98268bcae5-sid-1 {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 748fb97f697941489dd7ef4042a092d8 2023-10-15T16:04:50,953 DEBUG [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.StoreEngine(279): loaded hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/sid/748fb97f697941489dd7ef4042a092d8 2023-10-15T16:04:50,953 INFO [StoreOpener-2d525cde81451d9cc8ed5e98268bcae5-1 {}] regionserver.HStore(324): Store=2d525cde81451d9cc8ed5e98268bcae5/sid, memstore type=DefaultMemStore, storagePolicy=NONE, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-10-15T16:04:50,954 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(1028): replaying wal for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,954 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,955 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(5511): Found 0 recovered edits file(s) under hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,956 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(1038): stopping wal replay for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,957 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(1050): Cleaning up temporary data for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,957 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:replication descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-10-15T16:04:50,959 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(1083): writing seq id for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,960 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(1104): Opened 2d525cde81451d9cc8ed5e98268bcae5; next sequenceid=35; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70293359, jitterRate=0.04745267331600189}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-10-15T16:04:50,960 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(1112): Running coprocessor post-open hooks for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:50,960 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegion(996): Region open journal for 2d525cde81451d9cc8ed5e98268bcae5: Running coprocessor pre-open hook at 1697385890873Writing region info on filesystem at 1697385890873Initializing all the Stores at 1697385890873Instantiating store for column family {NAME => 'hfileref', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385890874 (+1 ms)Instantiating store for column family {NAME => 'queue', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385890880 (+6 ms)Instantiating store for column family {NAME => 'sid', INDEX_BLOCK_ENCODING => 'NONE', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536 B (64KB)'} at 1697385890880Cleaning up temporary data from old regions at 1697385890957 (+77 ms)Running coprocessor post-open hooks at 1697385890960 (+3 ms)Region opened successfully at 1697385890960 2023-10-15T16:04:50,961 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegionServer(2186): Post open deploy tasks for hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., pid=61, masterSystemTime=1697385890812 2023-10-15T16:04:50,963 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.CompactSplit(403): Add compact mark for store 2d525cde81451d9cc8ed5e98268bcae5:queue, priority=-2147483648, current under compaction store size is 1 2023-10-15T16:04:50,964 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.CompactSplit(411): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-15T16:04:50,964 DEBUG [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-10-15T16:04:50,970 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] regionserver.HRegionServer(2216): Finished post open deploy task for hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:50,970 INFO [RS_OPEN_PRIORITY_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_OPEN_PRIORITY_REGION, pid=61}] handler.AssignRegionHandler(147): Opened hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:50,971 INFO [PEWorker-4 {}] assignment.RegionStateStore(223): pid=59 updating hbase:meta row=2d525cde81451d9cc8ed5e98268bcae5, regionState=OPEN, openSeqNum=35, regionLocation=8c5d72df62de,35419,1697385888915 2023-10-15T16:04:50,972 DEBUG [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 18105 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-10-15T16:04:50,974 DEBUG [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] regionserver.HStore(1535): 2d525cde81451d9cc8ed5e98268bcae5/queue is initiating minor compaction (all files) 2023-10-15T16:04:50,974 INFO [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] regionserver.HRegion(2399): Starting compaction of 2d525cde81451d9cc8ed5e98268bcae5/queue in hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:50,974 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=61, resume processing ppid=59 2023-10-15T16:04:50,974 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=61, ppid=59, state=SUCCESS, hasLock=false; OpenRegionProcedure 2d525cde81451d9cc8ed5e98268bcae5, server=8c5d72df62de,35419,1697385888915 in 314 msec 2023-10-15T16:04:50,974 INFO [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] regionserver.HStore(1171): Starting compaction of [hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/46fdfd784a1242dd9339a7a4fc97d13a, hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/8807426219fd436c817c02ff77937323, hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/d0ec0dbb35af4b308ac053911182fd96] into tmpdir=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp, totalSize=17.7 K 2023-10-15T16:04:50,975 DEBUG [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] compactions.Compactor(224): Compacting 46fdfd784a1242dd9339a7a4fc97d13a, keycount=6, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=14, earliestPutTs=1697385867222 2023-10-15T16:04:50,976 DEBUG [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] compactions.Compactor(224): Compacting 8807426219fd436c817c02ff77937323, keycount=4, bloomtype=ROW, size=5.8 K, encoding=NONE, compression=NONE, seqNum=22, earliestPutTs=1697385878972 2023-10-15T16:04:50,976 DEBUG [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] compactions.Compactor(224): Compacting d0ec0dbb35af4b308ac053911182fd96, keycount=6, bloomtype=ROW, size=6.1 K, encoding=NONE, compression=NONE, seqNum=31, earliestPutTs=1697385881607 2023-10-15T16:04:50,976 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=59, resume processing ppid=56 2023-10-15T16:04:50,976 INFO [PEWorker-2 {}] procedure2.ProcedureExecutor(1413): Finished pid=59, ppid=56, state=SUCCESS, hasLock=false; TransitRegionStateProcedure table=hbase:replication, region=2d525cde81451d9cc8ed5e98268bcae5, ASSIGN in 587 msec 2023-10-15T16:04:50,976 INFO [PEWorker-5 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=63, ppid=56, state=RUNNABLE:ASSIGN_REPLICATION_QUEUES_ADD_MISSING_QUEUES, hasLock=false; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure}] 2023-10-15T16:04:50,979 DEBUG [PEWorker-3 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:46527 this server is in the failed servers list 2023-10-15T16:04:50,980 DEBUG [PEWorker-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,46527,1697385859222, seqNum=23 , the old value is region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,46527,1697385859222, seqNum=23, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:46527 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:46527 2023-10-15T16:04:50,980 DEBUG [PEWorker-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,46527,1697385859222, seqNum=23 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:46527 2023-10-15T16:04:50,980 DEBUG [PEWorker-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,46527,1697385859222, seqNum=23 from cache 2023-10-15T16:04:51,008 INFO [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] throttle.PressureAwareThroughputController(145): 2d525cde81451d9cc8ed5e98268bcae5#queue#compaction#21 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-10-15T16:04:51,010 DEBUG [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/queue/c1c50593dc49434f8a6bdf42b691cb43 is 193, key is 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285/queue:8c5d72df62de%2C45999%2C1697385859285.rep/1697385884956/Put/seqid=0 2023-10-15T16:04:51,043 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:04:51,052 DEBUG [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/.tmp/queue/c1c50593dc49434f8a6bdf42b691cb43 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/c1c50593dc49434f8a6bdf42b691cb43 2023-10-15T16:04:51,068 INFO [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] regionserver.HStore(1332): Completed compaction of 3 (all) file(s) in 2d525cde81451d9cc8ed5e98268bcae5/queue of 2d525cde81451d9cc8ed5e98268bcae5 into c1c50593dc49434f8a6bdf42b691cb43(size=5.8 K), total size for store is 5.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-10-15T16:04:51,068 DEBUG [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] regionserver.HRegion(2429): Compaction status journal for 2d525cde81451d9cc8ed5e98268bcae5: 2023-10-15T16:04:51,068 INFO [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] regionserver.CompactSplit$CompactionRunner(669): Completed compaction region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., storeName=2d525cde81451d9cc8ed5e98268bcae5/queue, priority=13, startTime=1697385890963; duration=0sec 2023-10-15T16:04:51,069 DEBUG [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] regionserver.CompactSplit$CompactionRunner(701): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-10-15T16:04:51,069 DEBUG [RS:3;8c5d72df62de:35419-shortCompactions-0 {}] regionserver.CompactSplit$CompactionRunner(719): Remove under compaction mark for store: 2d525cde81451d9cc8ed5e98268bcae5:queue 2023-10-15T16:04:51,086 DEBUG [HBase-Metrics2-1 {}] impl.GlobalMetricRegistriesAdapter(131): Registering adapter for the MetricRegistry: RegionServer,sub=TableRequests_Namespace_hbase_table_replication 2023-10-15T16:04:51,087 INFO [HBase-Metrics2-1 {}] impl.GlobalMetricRegistriesAdapter(135): Registering RegionServer,sub=TableRequests_Namespace_hbase_table_replication Metrics about Tables on a single HBase RegionServer 2023-10-15T16:04:51,108 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'hbase:replication', row='', locateType=CURRENT is [region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,35419,1697385888915, seqNum=35] 2023-10-15T16:04:51,133 DEBUG [PEWorker-3 {}] replication.AssignReplicationQueuesProcedure(120): There are 2 replication queues need to be claimed for 8c5d72df62de,46527,1697385859222 2023-10-15T16:04:51,134 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.ClaimReplicationQueueRemoteProcedure}] 2023-10-15T16:04:51,293 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35419 {}] regionserver.RSRpcServices(3882): Executing remote procedure class org.apache.hadoop.hbase.replication.regionserver.ClaimReplicationQueueCallable, pid=64 2023-10-15T16:04:51,317 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 0, reset compression=false 2023-10-15T16:04:51,322 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=0, beingWritten=true 2023-10-15T16:04:51,327 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.ReplicationSourceWALReader(175): Read 0 WAL entries eligible for replication 2023-10-15T16:04:51,327 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 383, reset compression=false 2023-10-15T16:04:51,327 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.shipper8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,35419,1697385888915 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, lastWalPosition=383, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-10-15T16:04:51,327 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.shipper8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:04:51,328 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.shipper8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:04:51,328 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.shipper8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] ipc.RpcConnection(127): Using SIMPLE authentication for service=ClientService, sasl=false 2023-10-15T16:04:51,330 INFO [MiniHBaseClusterRegionServer-EventLoopGroup-13-3 {}] ipc.ServerRpcConnection(478): Connection from 172.17.0.3:38649, version=4.0.0-alpha-1-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=ClientService 2023-10-15T16:04:51,332 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=383, beingWritten=true 2023-10-15T16:04:51,336 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-2 {}] client.AsyncNonMetaRegionLocator(310): The fetched location of 'hbase:replication', row='1-8c5d72df62de,35419,1697385888915', locateType=CURRENT is [region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,35419,1697385888915, seqNum=35] 2023-10-15T16:04:51,538 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 383, reset compression=false 2023-10-15T16:04:51,543 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=383, beingWritten=true 2023-10-15T16:04:51,848 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 383, reset compression=false 2023-10-15T16:04:51,865 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=383, beingWritten=true 2023-10-15T16:04:52,043 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:04:52,277 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 383, reset compression=false 2023-10-15T16:04:52,306 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=383, beingWritten=true 2023-10-15T16:04:52,812 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 383, reset compression=false 2023-10-15T16:04:52,815 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=383, beingWritten=true 2023-10-15T16:04:53,043 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:04:53,419 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 383, reset compression=false 2023-10-15T16:04:53,424 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=383, beingWritten=true 2023-10-15T16:04:53,673 WARN [HBase-Metrics2-1 {}] impl.MetricsConfig(136): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-10-15T16:04:54,031 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] regionserver.Replication$ReplicationStatisticsChore(208): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B Normal source for cluster 1: Total replicated edits: 0, current progress: walGroup [8c5d72df62de%2C35419%2C1697385888915]: currently replicating from: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 at position: 383 2023-10-15T16:04:54,043 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:04:54,128 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 383, reset compression=false 2023-10-15T16:04:54,136 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=383, beingWritten=true 2023-10-15T16:04:54,585 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=64}] regionserver.ReplicationSource(231): queueId=1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222, ReplicationSource: 1, currentBandwidth=0 2023-10-15T16:04:54,588 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=64}] wal.AbstractFSWALProvider(366): Seprated old WAL dir hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de,46527,1697385859222 not exists 2023-10-15T16:04:54,588 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=64}] regionserver.ReplicationSourceManager(843): Skip enqueuing log 8c5d72df62de%2C46527%2C1697385859222.1697385861048 because it is before the start offset 8c5d72df62de%2C46527%2C1697385859222.1697385867323:383 2023-10-15T16:04:54,588 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=64}] regionserver.ReplicationSourceManager(843): Skip enqueuing log 8c5d72df62de%2C46527%2C1697385859222.meta.1697385880058.meta because it is before the start offset null 2023-10-15T16:04:54,588 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=64}] regionserver.ReplicationSourceManager(888): Added source for recovered queue 1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222, number of wals to replicate: 2 2023-10-15T16:04:54,588 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=64}] regionserver.ReplicationSourceManager(891): Enqueueing log hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C46527%2C1697385859222.1697385867323 from recovered queue for source: 1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 2023-10-15T16:04:54,588 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=64}] regionserver.ReplicationSourceManager(891): Enqueueing log hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C46527%2C1697385859222.rep.1697385880933 from recovered queue for source: 1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 2023-10-15T16:04:54,589 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0 {event_type=RS_CLAIM_REPLICATION_QUEUE, pid=64}] regionserver.RemoteProcedureResultReporter(61): Successfully complete execution of pid=64 2023-10-15T16:04:54,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(4067): Remote procedure done, pid=64 2023-10-15T16:04:54,593 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=64, resume processing ppid=63 2023-10-15T16:04:54,593 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1413): Finished pid=64, ppid=63, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.ClaimReplicationQueueRemoteProcedure in 3.4570 sec 2023-10-15T16:04:54,602 DEBUG [PEWorker-4 {}] replication.AssignReplicationQueuesProcedure(120): There are 1 replication queues need to be claimed for 8c5d72df62de,46527,1697385859222 2023-10-15T16:04:54,602 INFO [PEWorker-4 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=65, ppid=63, state=RUNNABLE, hasLock=false; org.apache.hadoop.hbase.master.replication.ClaimReplicationQueueRemoteProcedure}] 2023-10-15T16:04:54,609 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] zookeeper.RecoverableZooKeeper(141): Process identifier=connection to cluster: 1 connecting to ZooKeeper ensemble=127.0.0.1:52753 2023-10-15T16:04:54,613 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] zookeeper.ReadOnlyZKClient(139): Connect 0x0a7feabd to 127.0.0.1:52753 with session timeout=90000ms, retries 1, retry interval 10ms, keepAlive=60000ms 2023-10-15T16:04:54,613 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 10x0, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-10-15T16:04:54,620 DEBUG [zk-event-processor-pool-0 {}] zookeeper.ZKWatcher(630): connection to cluster: 1-0x101b6f97f510033 connected 2023-10-15T16:04:54,629 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] ipc.AbstractRpcClient(197): Codec=org.apache.hadoop.hbase.codec.KeyValueCodecWithTags@2972f969, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-10-15T16:04:54,630 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSource(604): peerId=1, queueId=1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 (queues=2) is replicating from cluster=b3f2b9ba-d33e-407e-b223-f02b5843aee1 to cluster=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:04:54,630 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C46527%2C1697385859222.rep 2023-10-15T16:04:54,630 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:54,644 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C46527%2C1697385859222.rep.1697385880933, startPosition=0, beingWritten=false 2023-10-15T16:04:54,649 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSource(369): peerId=1, starting shipping worker for walGroupId=8c5d72df62de%2C46527%2C1697385859222 2023-10-15T16:04:54,649 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C46527%2C1697385859222.rep 2023-10-15T16:04:54,649 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceWALReader(109): peerClusterZnode=1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222, ReplicationSourceWALReaderThread : 1 inited, replicationBatchSizeCapacity=102400, replicationBatchCountCapacity=25000, replicationBatchQueueCapacity=1 2023-10-15T16:04:54,651 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C46527%2C1697385859222.1697385867323, startPosition=383, beingWritten=false 2023-10-15T16:04:54,654 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(451): EOF, closing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C46527%2C1697385859222.rep.1697385880933 2023-10-15T16:04:54,654 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceWALReader(175): Read 0 WAL entries eligible for replication 2023-10-15T16:04:54,654 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(236): No more WAL files in queue 2023-10-15T16:04:54,654 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceWALReader(118): Stopping the replication source wal reader 2023-10-15T16:04:54,654 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C46527%2C1697385859222.rep.1697385880933, lastWalPosition=3060, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-10-15T16:04:54,658 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceManager(720): Removing 1 logs in the list: [8c5d72df62de%2C46527%2C1697385859222.rep.1697385880933] 2023-10-15T16:04:54,658 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceManager(734): Removing 0 logs from remote dir hdfs://localhost:40215/user/jenkins/test-data/992102bd-50e5-ed7b-f035-a525dec54203/remoteWALs in the list: [] 2023-10-15T16:04:54,658 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=null, lastWalPosition=0, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=false,usedBufferSize=0] 2023-10-15T16:04:54,658 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(451): EOF, closing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C46527%2C1697385859222.1697385867323 2023-10-15T16:04:54,658 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(138): Finished recovering queue for group 8c5d72df62de%2C46527%2C1697385859222.rep of peer 1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 2023-10-15T16:04:54,659 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceWALReader(175): Read 0 WAL entries eligible for replication 2023-10-15T16:04:54,659 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceManager(540): Done with the recovered queue 1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 2023-10-15T16:04:54,659 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.WALEntryStream(236): No more WAL files in queue 2023-10-15T16:04:54,659 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.wal-reader.8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceWALReader(118): Stopping the replication source wal reader 2023-10-15T16:04:54,661 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222.rep,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceManager(553): Finished recovering queue 1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 with the following stats: Total replicated edits: 0, current progress: 2023-10-15T16:04:54,664 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(98): Running ReplicationSourceShipper Thread for wal group: 8c5d72df62de%2C46527%2C1697385859222 2023-10-15T16:04:54,664 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSourceShipper(110): Shipper from source 1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 got entry batch from reader: WALEntryBatch [walEntries=[], lastWalPath=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/oldWALs/8c5d72df62de%2C46527%2C1697385859222.1697385867323, lastWalPosition=648, nbRowKeys=0, nbHFiles=0, heapSize=0, lastSeqIds={}, endOfFile=true,usedBufferSize=0] 2023-10-15T16:04:54,666 ERROR [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSource(452): Unexpected exception in RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 currentPath=null java.lang.NullPointerException: null at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.cleanOldLogs(ReplicationSourceManager.java:662) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:649) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceInterface.logPositionAndCleanOldLogs(ReplicationSourceInterface.java:211) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.updateLogPosition(ReplicationSourceShipper.java:266) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:158) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:119) ~[classes/:?] 2023-10-15T16:04:54,666 ERROR [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.HRegionServer(2389): ***** ABORTING region server 8c5d72df62de,35419,1697385888915: Unexpected exception in RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 ***** java.lang.NullPointerException: null at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.cleanOldLogs(ReplicationSourceManager.java:662) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:649) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceInterface.logPositionAndCleanOldLogs(ReplicationSourceInterface.java:211) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.updateLogPosition(ReplicationSourceShipper.java:266) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:158) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:119) ~[classes/:?] 2023-10-15T16:04:54,666 ERROR [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.HRegionServer(2396): RegionServer abort: loaded coprocessors are: [org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint] 2023-10-15T16:04:54,667 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] util.JSONBean(135): Listing beans for java.lang:type=Memory 2023-10-15T16:04:54,668 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] util.JSONBean(135): Listing beans for Hadoop:service=HBase,name=RegionServer,sub=IPC 2023-10-15T16:04:54,673 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] util.JSONBean(135): Listing beans for Hadoop:service=HBase,name=RegionServer,sub=Replication 2023-10-15T16:04:54,673 DEBUG [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] util.JSONBean(135): Listing beans for Hadoop:service=HBase,name=RegionServer,sub=Server 2023-10-15T16:04:54,673 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.HRegionServer(2400): Dump of metrics as JSON on abort: { "beans": [ { "name": "java.lang:type=Memory", "modelerType": "sun.management.MemoryImpl", "Verbose": false, "ObjectPendingFinalizationCount": 0, "HeapMemoryUsage": { "committed": 1048576000, "init": 1048576000, "max": 2306867200, "used": 756523008 }, "NonHeapMemoryUsage": { "committed": 197238784, "init": 7667712, "max": -1, "used": 193158536 }, "ObjectName": "java.lang:type=Memory" } ], "beans": [], "beans": [], "beans": [] } 2023-10-15T16:04:54,674 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33403 {}] master.MasterRpcServices(697): 8c5d72df62de,35419,1697385888915 reported a fatal error: ***** ABORTING region server 8c5d72df62de,35419,1697385888915: Unexpected exception in RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 ***** Cause: java.lang.NullPointerException at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.cleanOldLogs(ReplicationSourceManager.java:662) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.logPositionAndCleanOldLogs(ReplicationSourceManager.java:649) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceInterface.logPositionAndCleanOldLogs(ReplicationSourceInterface.java:211) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.updateLogPosition(ReplicationSourceShipper.java:266) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.shipEdits(ReplicationSourceShipper.java:158) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:119) 2023-10-15T16:04:54,719 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.HRegionServer(2147): ***** STOPPING region server '8c5d72df62de,35419,1697385888915' ***** 2023-10-15T16:04:54,719 INFO [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.HRegionServer(2161): STOPPED: Unexpected exception in RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 2023-10-15T16:04:54,719 WARN [RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222.replicationSource.shipper8c5d72df62de%2C46527%2C1697385859222,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222 {}] regionserver.ReplicationSource(462): Server is shutting down, give up refreshing source for peer 1 2023-10-15T16:04:54,721 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HeapMemoryManager(220): Stopping 2023-10-15T16:04:54,721 INFO [RS:3;8c5d72df62de:35419 {}] flush.RegionServerFlushTableProcedureManager(119): Stopping region server flush procedure manager abruptly. 2023-10-15T16:04:54,721 INFO [RS:3;8c5d72df62de:35419 {}] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager abruptly. 2023-10-15T16:04:54,721 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(3035): Received CLOSE for 2d525cde81451d9cc8ed5e98268bcae5 2023-10-15T16:04:54,721 INFO [MemStoreFlusher.0 {}] regionserver.MemStoreFlusher$FlushHandler(374): MemStoreFlusher.0 exiting 2023-10-15T16:04:54,722 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(3035): Received CLOSE for f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:54,722 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(940): aborting server 8c5d72df62de,35419,1697385888915 2023-10-15T16:04:54,722 INFO [RS:3;8c5d72df62de:35419 {}] hbase.HBaseServerBase(452): Close async cluster connection 2023-10-15T16:04:54,722 INFO [RS:3;8c5d72df62de:35419 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:3;8c5d72df62de:35419. 2023-10-15T16:04:54,722 DEBUG [RS:3;8c5d72df62de:35419 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.HBaseServerBase.closeClusterConnection(HBaseServerBase.java:454) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:946) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:54,722 DEBUG [RS:3;8c5d72df62de:35419 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:54,723 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.CompactSplit(469): Waiting for Split Thread to finish... 2023-10-15T16:04:54,723 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.CompactSplit(469): Waiting for Large Compaction Thread to finish... 2023-10-15T16:04:54,723 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.CompactSplit(469): Waiting for Small Compaction Thread to finish... 2023-10-15T16:04:54,723 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(3035): Received CLOSE for 1588230740 2023-10-15T16:04:54,723 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1710): Closing 2d525cde81451d9cc8ed5e98268bcae5, disabling compactions & flushes 2023-10-15T16:04:54,723 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1743): Closing region hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:54,723 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1764): Time limited wait for close lock on hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:54,723 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1831): Acquired close lock on hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. after waiting 0 ms 2023-10-15T16:04:54,723 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1841): Updates disabled for region hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:54,726 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(1296): Waiting on 3 regions to close 2023-10-15T16:04:54,727 DEBUG [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(1300): Online Regions={1588230740=hbase:meta,,1.1588230740, 2d525cde81451d9cc8ed5e98268bcae5=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., f3bcdec0351888cb7677be57c1f453c1=SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1.} 2023-10-15T16:04:54,727 DEBUG [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(1326): Waiting on 1588230740, 2d525cde81451d9cc8ed5e98268bcae5, f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:54,759 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1710): Closing 1588230740, disabling compactions & flushes 2023-10-15T16:04:54,759 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1743): Closing region hbase:meta,,1.1588230740 2023-10-15T16:04:54,759 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1764): Time limited wait for close lock on hbase:meta,,1.1588230740 2023-10-15T16:04:54,759 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1831): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-10-15T16:04:54,759 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1841): Updates disabled for region hbase:meta,,1.1588230740 2023-10-15T16:04:54,769 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-3 {}] ipc.MetricsHBaseServer(152): Unknown exception type org.apache.hadoop.hbase.regionserver.RegionServerAbortedException: Server 8c5d72df62de,35419,1697385888915 aborting at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processRequest(ServerRpcConnection.java:558) ~[classes/:?] at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processOneRpc(ServerRpcConnection.java:358) ~[classes/:?] at org.apache.hadoop.hbase.ipc.NettyServerRpcConnection.process(NettyServerRpcConnection.java:89) ~[classes/:?] at org.apache.hadoop.hbase.ipc.NettyRpcServerRequestDecoder.channelRead0(NettyRpcServerRequestDecoder.java:56) ~[classes/:?] at org.apache.hadoop.hbase.ipc.NettyRpcServerRequestDecoder.channelRead0(NettyRpcServerRequestDecoder.java:31) ~[classes/:?] at org.apache.hbase.thirdparty.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:54,784 DEBUG [RSProcedureDispatcher-pool-0 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(271): Request to 8c5d72df62de,35419,1697385888915 failed, try=0 org.apache.hadoop.hbase.regionserver.RegionServerAbortedException: Server 8c5d72df62de,35419,1697385888915 aborting at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:110) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:100) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.unwrapException(RSProcedureDispatcher.java:384) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:400) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: Server 8c5d72df62de,35419,1697385888915 aborting at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:186) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.sendRequest(RSProcedureDispatcher.java:431) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:398) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:391) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:197) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:218) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:04:54,786 WARN [RSProcedureDispatcher-pool-0 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(314): 8c5d72df62de,35419,1697385888915 is aborted or stopped, for safety we still need to wait until it is fully dead, try=0 2023-10-15T16:04:54,810 INFO [regionserver/8c5d72df62de:0.leaseChecker {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:54,822 DEBUG [StoreCloser-hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.-1 {}] regionserver.HStore(2307): Moving the files [hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/46fdfd784a1242dd9339a7a4fc97d13a, hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/8807426219fd436c817c02ff77937323, hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/d0ec0dbb35af4b308ac053911182fd96] to archive 2023-10-15T16:04:54,835 DEBUG [StoreCloser-hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.-1 {}] backup.HFileArchiver(360): Archiving compacted files. 2023-10-15T16:04:54,858 ERROR [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1948): Memstore data size is 1931 in region hbase:meta,,1.1588230740 2023-10-15T16:04:54,858 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-15T16:04:54,859 INFO [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1961): Closed hbase:meta,,1.1588230740 2023-10-15T16:04:54,859 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] regionserver.HRegion(1664): Region close journal for 1588230740: Waiting for close lock at 1697385894727Running coprocessor pre-close hooks at 1697385894727Disabling compacts and flushes for region at 1697385894727Disabling writes for close at 1697385894759 (+32 ms)Writing region close event to WAL at 1697385894858 (+99 ms)Running coprocessor post-close hooks at 1697385894858Closed at 1697385894858 2023-10-15T16:04:54,859 DEBUG [RS_CLOSE_META-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_META}] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-10-15T16:04:54,860 DEBUG [StoreCloser-hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.-1 {}] backup.HFileArchiver(596): Archived from FileableStoreFile, hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/46fdfd784a1242dd9339a7a4fc97d13a to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/archive/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/46fdfd784a1242dd9339a7a4fc97d13a 2023-10-15T16:04:54,861 DEBUG [StoreCloser-hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.-1 {}] backup.HFileArchiver(596): Archived from FileableStoreFile, hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/8807426219fd436c817c02ff77937323 to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/archive/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/8807426219fd436c817c02ff77937323 2023-10-15T16:04:54,863 DEBUG [StoreCloser-hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.-1 {}] backup.HFileArchiver(596): Archived from FileableStoreFile, hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/d0ec0dbb35af4b308ac053911182fd96 to hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/archive/data/hbase/replication/2d525cde81451d9cc8ed5e98268bcae5/queue/d0ec0dbb35af4b308ac053911182fd96 2023-10-15T16:04:54,884 DEBUG [StoreCloser-hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.-1 {}] regionserver.HRegionServer(3476): Failed to report file archival(s) to Master. This will be retried. org.apache.hadoop.hbase.ipc.StoppedRpcClientException: Call to address=8c5d72df62de:33403 failed on local exception: org.apache.hadoop.hbase.ipc.StoppedRpcClientException at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:217) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:335) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$200(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:599) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.reportFileArchival(RegionServerStatusProtos.java:18029) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.regionserver.HRegionServer.reportFileArchivalForQuotas(HRegionServer.java:3460) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.HStore.reportArchivedFilesForQuota(HStore.java:2402) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.HStore.removeCompactedfiles(HStore.java:2337) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.HStore.closeWithoutLock(HStore.java:734) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.HStore.close(HStore.java:800) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.HRegion$2.call(HRegion.java:1900) ~[classes/:?] at org.apache.hadoop.hbase.regionserver.HRegion$2.call(HRegion.java:1897) ~[classes/:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.StoppedRpcClientException at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:365) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 17 more 2023-10-15T16:04:54,885 WARN [StoreCloser-hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5.-1 {}] regionserver.HStore(2404): Failed to report archival of files: [46fdfd784a1242dd9339a7a4fc97d13a=5952, 8807426219fd436c817c02ff77937323=5951, d0ec0dbb35af4b308ac053911182fd96=6202] 2023-10-15T16:04:54,887 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-3 {}] ipc.MetricsHBaseServer(152): Unknown exception type org.apache.hadoop.hbase.regionserver.RegionServerAbortedException: Server 8c5d72df62de,35419,1697385888915 aborting at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processRequest(ServerRpcConnection.java:558) ~[classes/:?] at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processOneRpc(ServerRpcConnection.java:358) ~[classes/:?] at org.apache.hadoop.hbase.ipc.NettyServerRpcConnection.process(NettyServerRpcConnection.java:89) ~[classes/:?] at org.apache.hadoop.hbase.ipc.NettyRpcServerRequestDecoder.channelRead0(NettyRpcServerRequestDecoder.java:56) ~[classes/:?] at org.apache.hadoop.hbase.ipc.NettyRpcServerRequestDecoder.channelRead0(NettyRpcServerRequestDecoder.java:31) ~[classes/:?] at org.apache.hbase.thirdparty.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:54,896 DEBUG [RSProcedureDispatcher-pool-2 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(271): Request to 8c5d72df62de,35419,1697385888915 failed, try=1 org.apache.hadoop.hbase.regionserver.RegionServerAbortedException: Server 8c5d72df62de,35419,1697385888915 aborting at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:110) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:100) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.unwrapException(RSProcedureDispatcher.java:384) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:400) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: Server 8c5d72df62de,35419,1697385888915 aborting at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:186) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.sendRequest(RSProcedureDispatcher.java:431) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:398) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:391) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:197) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:218) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:04:54,898 WARN [RSProcedureDispatcher-pool-2 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(314): 8c5d72df62de,35419,1697385888915 is aborted or stopped, for safety we still need to wait until it is fully dead, try=1 2023-10-15T16:04:54,912 ERROR [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1948): Memstore data size is 852 in region hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:54,912 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-15T16:04:54,913 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1961): Closed hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:54,913 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1664): Region close journal for 2d525cde81451d9cc8ed5e98268bcae5: Waiting for close lock at 1697385894723Running coprocessor pre-close hooks at 1697385894723Disabling compacts and flushes for region at 1697385894723Disabling writes for close at 1697385894723Writing region close event to WAL at 1697385894912 (+189 ms)Running coprocessor post-close hooks at 1697385894912Closed at 1697385894913 (+1 ms) 2023-10-15T16:04:54,913 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] handler.CloseRegionHandler(117): Closed hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5. 2023-10-15T16:04:54,914 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1710): Closing f3bcdec0351888cb7677be57c1f453c1, disabling compactions & flushes 2023-10-15T16:04:54,914 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1743): Closing region SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:54,914 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1764): Time limited wait for close lock on SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:54,914 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1831): Acquired close lock on SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. after waiting 0 ms 2023-10-15T16:04:54,914 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1841): Updates disabled for region SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:54,927 DEBUG [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(1326): Waiting on f3bcdec0351888cb7677be57c1f453c1 2023-10-15T16:04:54,931 INFO [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1961): Closed SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:54,932 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] regionserver.HRegion(1664): Region close journal for f3bcdec0351888cb7677be57c1f453c1: Waiting for close lock at 1697385894913Running coprocessor pre-close hooks at 1697385894913Disabling compacts and flushes for region at 1697385894913Disabling writes for close at 1697385894914 (+1 ms)Writing region close event to WAL at 1697385894931 (+17 ms)Running coprocessor post-close hooks at 1697385894931Closed at 1697385894931 2023-10-15T16:04:54,932 DEBUG [RS_CLOSE_REGION-regionserver/8c5d72df62de:0-0 {event_type=M_RS_CLOSE_REGION}] handler.CloseRegionHandler(117): Closed SyncRep,,1697385862926.f3bcdec0351888cb7677be57c1f453c1. 2023-10-15T16:04:54,943 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(213): Reset reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160 to pos 383, reset compression=false 2023-10-15T16:04:54,961 DEBUG [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.WALEntryStream(249): Creating new reader hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.1697385889160, startPosition=383, beingWritten=true 2023-10-15T16:04:55,002 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] hbase.ScheduledChore(149): Chore: CompactionChecker was stopped 2023-10-15T16:04:55,002 INFO [regionserver/8c5d72df62de:0.Chore.1 {}] hbase.ScheduledChore(149): Chore: MemstoreFlusherChore was stopped 2023-10-15T16:04:55,044 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:04:55,128 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(960): stopping server 8c5d72df62de,35419,1697385888915; all regions closed. 2023-10-15T16:04:55,135 WARN [Close-WAL-Writer-0 {}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(596): complete file /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915/8c5d72df62de%2C35419%2C1697385888915.rep.1697385890819 not finished, retry = 0 2023-10-15T16:04:55,241 DEBUG [RS:3;8c5d72df62de:35419 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:55,241 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.LeaseManager(133): Closed leases 2023-10-15T16:04:55,241 INFO [RS:3;8c5d72df62de:35419 {}] hbase.HBaseServerBase(435): Shutdown chores and chore service 2023-10-15T16:04:55,242 INFO [RS:3;8c5d72df62de:35419 {}] hbase.ChoreService(370): Chore service for: regionserver/8c5d72df62de:0 had [ScheduledChore name=ReplicationSinkStatistics, period=5000, unit=MILLISECONDS, ScheduledChore name=ReplicationSourceStatistics, period=5000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-10-15T16:04:55,242 INFO [RS:3;8c5d72df62de:35419 {}] hbase.HBaseServerBase(445): Shutdown executor service 2023-10-15T16:04:55,242 INFO [regionserver/8c5d72df62de:0.logRoller {}] wal.AbstractWALRoller(249): LogRoller exiting. 2023-10-15T16:04:55,242 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.ReplicationSource(674): peerId=1, Closing source 1-8c5d72df62de,35419,1697385888915 because: Region server is closing 2023-10-15T16:04:55,299 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-13-3 {}] ipc.MetricsHBaseServer(152): Unknown exception type org.apache.hadoop.hbase.regionserver.RegionServerAbortedException: Server 8c5d72df62de,35419,1697385888915 aborting at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processRequest(ServerRpcConnection.java:558) ~[classes/:?] at org.apache.hadoop.hbase.ipc.ServerRpcConnection.processOneRpc(ServerRpcConnection.java:358) ~[classes/:?] at org.apache.hadoop.hbase.ipc.NettyServerRpcConnection.process(NettyServerRpcConnection.java:89) ~[classes/:?] at org.apache.hadoop.hbase.ipc.NettyRpcServerRequestDecoder.channelRead0(NettyRpcServerRequestDecoder.java:56) ~[classes/:?] at org.apache.hadoop.hbase.ipc.NettyRpcServerRequestDecoder.channelRead0(NettyRpcServerRequestDecoder.java:31) ~[classes/:?] at org.apache.hbase.thirdparty.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:55,300 DEBUG [RSProcedureDispatcher-pool-1 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(271): Request to 8c5d72df62de,35419,1697385888915 failed, try=2 org.apache.hadoop.hbase.regionserver.RegionServerAbortedException: Server 8c5d72df62de,35419,1697385888915 aborting at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:110) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:100) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.unwrapException(RSProcedureDispatcher.java:384) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:400) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException: Server 8c5d72df62de,35419,1697385888915 aborting at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:186) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.sendRequest(RSProcedureDispatcher.java:431) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:398) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:391) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:197) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:218) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:509) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:04:55,301 WARN [RSProcedureDispatcher-pool-1 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(314): 8c5d72df62de,35419,1697385888915 is aborted or stopped, for safety we still need to wait until it is fully dead, try=2 2023-10-15T16:04:55,345 DEBUG [RS:3;8c5d72df62de:35419.replicationSource,1-8c5d72df62de,35419,1697385888915-EventThread {}] zookeeper.ZKWatcher(604): connection to cluster: 1-0x101b6f97f510031, quorum=127.0.0.1:52753, baseZNode=/cluster1 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:55,345 INFO [RS:3;8c5d72df62de:35419 {}] client.AsyncConnectionImpl(233): Connection has been closed by RS:3;8c5d72df62de:35419. 2023-10-15T16:04:55,345 DEBUG [RS:3;8c5d72df62de:35419 {}] client.AsyncConnectionImpl(264): Call stack: at java.base/java.lang.Thread.getStackTrace(Thread.java:1602) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.lambda$close$5(AsyncConnectionImpl.java:235) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177) at org.apache.hadoop.hbase.client.AsyncConnectionImpl.close(AsyncConnectionImpl.java:229) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.disconnect(HBaseReplicationEndpoint.java:117) at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.doStop(HBaseInterClusterReplicationEndpoint.java:521) at org.apache.hbase.thirdparty.com.google.common.util.concurrent.AbstractService.stopAsync(AbstractService.java:285) at org.apache.hadoop.hbase.replication.HBaseReplicationEndpoint.stop(HBaseReplicationEndpoint.java:150) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:695) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:669) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:664) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.terminate(ReplicationSource.java:659) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:987) at org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:160) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2481) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:995) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(SingleProcessHBaseCluster.java:171) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.access$000(SingleProcessHBaseCluster.java:125) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer$1.run(SingleProcessHBaseCluster.java:155) at java.base/java.security.AccessController.doPrivileged(Native Method) at java.base/javax.security.auth.Subject.doAs(Subject.java:361) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1742) at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:322) at org.apache.hadoop.hbase.SingleProcessHBaseCluster$MiniHBaseClusterRegionServer.run(SingleProcessHBaseCluster.java:152) at java.base/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:04:55,345 DEBUG [RS:3;8c5d72df62de:35419 {}] ipc.AbstractRpcClient(513): Stopping rpc client 2023-10-15T16:04:55,345 DEBUG [RS:3;8c5d72df62de:35419 {}] zookeeper.ReadOnlyZKClient(365): Close zookeeper connection 0x5e0baefa to 127.0.0.1:52753 2023-10-15T16:04:55,446 WARN [RS:3;8c5d72df62de:35419.replicationSource.wal-reader.8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] util.Threads(127): sleep interrupted java.lang.InterruptedException: sleep interrupted at java.lang.Thread.sleep(Native Method) ~[?:?] at org.apache.hadoop.hbase.util.Threads.sleep(Threads.java:125) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.sleep(ReplicationSourceWALReader.java:128) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:160) ~[classes/:?] 2023-10-15T16:04:55,446 WARN [RS:3;8c5d72df62de:35419.replicationSource.shipper8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 {}] regionserver.ReplicationSourceShipper(123): Interrupted while waiting for next replication entry batch java.lang.InterruptedException: null at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?] at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?] at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) ~[?:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.poll(ReplicationSourceWALReader.java:307) ~[classes/:?] at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceShipper.run(ReplicationSourceShipper.java:109) ~[classes/:?] 2023-10-15T16:04:55,447 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.ReplicationSource(726): peerId=1, ReplicationSourceWorker RS:3;8c5d72df62de:35419.replicationSource.shipper8c5d72df62de%2C35419%2C1697385888915,1-8c5d72df62de,35419,1697385888915 terminated 2023-10-15T16:04:55,448 INFO [RS:3;8c5d72df62de:35419 {}] ipc.NettyRpcServer(340): Stopping server on /172.17.0.3:35419 2023-10-15T16:04:55,465 DEBUG [RegionServer-Restarter-EventThread {}] zookeeper.ZKWatcher(604): regionserver:35419-0x101b6f97f510030, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster2/rs/8c5d72df62de,35419,1697385888915 2023-10-15T16:04:55,465 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster2/rs 2023-10-15T16:04:55,465 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:38213-0x101b6f97f510005, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster2/rs 2023-10-15T16:04:55,465 INFO [RS:3;8c5d72df62de:35419 {}] hbase.HBaseServerBase(476): Close zookeeper 2023-10-15T16:04:55,466 INFO [RegionServerTracker-0 {}] master.RegionServerTracker(171): RegionServer ephemeral node deleted, processing expiration [8c5d72df62de,35419,1697385888915] 2023-10-15T16:04:55,469 DEBUG [RegionServerTracker-0 {}] zookeeper.RecoverableZooKeeper(221): Node /cluster2/draining/8c5d72df62de,35419,1697385888915 already deleted, retry=false 2023-10-15T16:04:55,469 INFO [RegionServerTracker-0 {}] master.ServerManager(604): Processing expiration of 8c5d72df62de,35419,1697385888915 on 8c5d72df62de,33403,1697385859093 2023-10-15T16:04:55,471 DEBUG [RegionServerTracker-0 {}] procedure2.ProcedureExecutor(1032): Stored pid=66, state=RUNNABLE:SERVER_CRASH_START, hasLock=false; ServerCrashProcedure 8c5d72df62de,35419,1697385888915, splitWal=true, meta=true 2023-10-15T16:04:55,471 INFO [RegionServerTracker-0 {}] assignment.AssignmentManager(1862): Scheduled ServerCrashProcedure pid=66 for 8c5d72df62de,35419,1697385888915 (carryingMeta=true) 8c5d72df62de,35419,1697385888915/CRASHED/regionCount=3/lock=java.util.concurrent.locks.ReentrantReadWriteLock@7d7df0b1[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-10-15T16:04:55,475 WARN [RegionServerTracker-0 {}] replication.ClaimReplicationQueueRemoteProcedure(121): Failed to claim replication queue 1-8c5d72df62de,46527,1697385859222/8c5d72df62de,45999,1697385859285 on server 8c5d72df62de,35419,1697385888915 org.apache.hadoop.hbase.DoNotRetryIOException: server not online 8c5d72df62de,35419,1697385888915 at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher.abortPendingOperations(RSProcedureDispatcher.java:163) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher.abortPendingOperations(RSProcedureDispatcher.java:61) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher$BufferNode.abortOperationsInQueue(RemoteProcedureDispatcher.java:417) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.removeNode(RemoteProcedureDispatcher.java:201) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher.serverRemoved(RSProcedureDispatcher.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.ServerManager.lambda$expireServer$3(ServerManager.java:609) ~[classes/:?] at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) ~[?:?] at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) ~[?:?] at org.apache.hadoop.hbase.master.ServerManager.expireServer(ServerManager.java:609) ~[classes/:?] at org.apache.hadoop.hbase.master.ServerManager.expireServer(ServerManager.java:563) ~[classes/:?] at org.apache.hadoop.hbase.master.RegionServerTracker.processAsActiveMaster(RegionServerTracker.java:172) ~[classes/:?] at org.apache.hadoop.hbase.master.RegionServerTracker.refresh(RegionServerTracker.java:206) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:55,476 INFO [PEWorker-3 {}] procedure.ServerCrashProcedure(163): Start pid=66, state=RUNNABLE:SERVER_CRASH_START, hasLock=true; ServerCrashProcedure 8c5d72df62de,35419,1697385888915, splitWal=true, meta=true 2023-10-15T16:04:55,476 INFO [master/8c5d72df62de:0.Chore.1 {}] master.ServerManager(1043): Rewriting .lastflushedseqids file at: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/.lastflushedseqids 2023-10-15T16:04:55,479 INFO [PEWorker-3 {}] procedure.ServerCrashProcedure(333): Splitting WALs pid=66, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, hasLock=true; ServerCrashProcedure 8c5d72df62de,35419,1697385888915, splitWal=true, meta=true, isMeta: true 2023-10-15T16:04:55,480 WARN [PEWorker-1 {}] procedure.ServerRemoteProcedure(98): Can not send remote operation 65 to 8c5d72df62de,35419,1697385888915, this operation will be retried to send to another server 2023-10-15T16:04:55,481 DEBUG [PEWorker-3 {}] master.MasterWalManager(317): Renamed region directory: hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/WALs/8c5d72df62de,35419,1697385888915-splitting 2023-10-15T16:04:55,483 INFO [PEWorker-3 {}] master.SplitWALManager(105): 8c5d72df62de,35419,1697385888915 WAL count=1, meta=true 2023-10-15T16:04:55,483 INFO [PEWorker-3 {}] procedure2.ProcedureExecutor(1684): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:ACQUIRE_SPLIT_WAL_WORKER, hasLock=false; SplitWALProcedure 8c5d72df62de%2C35419%2C1697385888915.meta.1697385890273.meta}] 2023-10-15T16:04:55,491 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1827): Finished subprocedure pid=65, resume processing ppid=63 2023-10-15T16:04:55,491 INFO [PEWorker-1 {}] procedure2.ProcedureExecutor(1413): Finished pid=65, ppid=63, state=SUCCESS, hasLock=false; org.apache.hadoop.hbase.master.replication.ClaimReplicationQueueRemoteProcedure in 878 msec 2023-10-15T16:04:55,491 WARN [PEWorker-1 {}] procedure2.RemoteProcedureDispatcher(184): since no node for this key 8c5d72df62de,35419,1697385888915, we can't removed the finished remote procedure 2023-10-15T16:04:55,494 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:55,495 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,35419,1697385888915, seqNum=35 , the old value is region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,35419,1697385888915, seqNum=35, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:04:55,495 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,35419,1697385888915, seqNum=35 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:04:55,495 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:replication,,1697385864935.2d525cde81451d9cc8ed5e98268bcae5., hostname=8c5d72df62de,35419,1697385888915, seqNum=35 from cache 2023-10-15T16:04:55,495 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:04:55,568 DEBUG [RegionServer-Restarter-EventThread {}] zookeeper.ZKWatcher(604): regionserver:35419-0x101b6f97f510030, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:55,568 INFO [RS:3;8c5d72df62de:35419 {}] hbase.HBaseServerBase(483): Close table descriptors 2023-10-15T16:04:55,568 DEBUG [RegionServer-Restarter-EventThread {}] zookeeper.ZKWatcher(604): regionserver:35419-0x101b6f97f510030, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=None, state=Closed, path=null 2023-10-15T16:04:55,568 INFO [RS:3;8c5d72df62de:35419 {}] regionserver.HRegionServer(1015): Exiting; stopping=8c5d72df62de,35419,1697385888915; zookeeper connection closed. 2023-10-15T16:04:55,569 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7cbbfe05 {}] hbase.SingleProcessHBaseCluster$SingleFileSystemShutdownThread(211): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7cbbfe05 2023-10-15T16:04:55,601 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:04:55,601 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:55,601 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:55,601 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:04:55,710 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:04:55,711 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:04:55,711 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:04:55,711 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:55,711 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:55,711 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:04:55,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:04:55,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:04:55,921 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:04:55,921 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:55,921 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:55,921 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:04:56,044 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:04:56,202 DEBUG [RSProcedureDispatcher-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:04:56,202 DEBUG [RSProcedureDispatcher-pool-0 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(271): Request to 8c5d72df62de,35419,1697385888915 failed, try=3 org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:186) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.sendRequest(RSProcedureDispatcher.java:431) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:398) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.AdminProtos$AdminService$Stub.executeProcedures(AdminProtos.java:35722) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRegionServerAdmin.lambda$executeProcedures$20(AsyncRegionServerAdmin.java:217) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRegionServerAdmin.call(AsyncRegionServerAdmin.java:102) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRegionServerAdmin.call(AsyncRegionServerAdmin.java:120) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRegionServerAdmin.executeProcedures(AsyncRegionServerAdmin.java:217) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.sendRequest(RSProcedureDispatcher.java:431) ~[classes/:?] at org.apache.hadoop.hbase.master.procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall.run(RSProcedureDispatcher.java:398) ~[classes/:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 12 more 2023-10-15T16:04:56,204 WARN [RSProcedureDispatcher-pool-0 {}] procedure.RSProcedureDispatcher$ExecuteProceduresRemoteCall(305): Request to 8c5d72df62de,35419,1697385888915 failed due to org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419, try=3 and the server is not online, give up 2023-10-15T16:04:56,204 INFO [RSProcedureDispatcher-pool-0 {}] procedure.ServerRemoteProcedure(132): This procedure 65 is already finished, skip the rest processes 2023-10-15T16:04:56,229 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:04:56,230 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:04:56,230 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:04:56,230 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:56,230 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:56,230 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:04:56,740 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:04:56,741 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:04:56,741 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:04:56,741 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:56,741 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:56,741 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:04:57,044 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:04:57,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:04:57,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:04:57,751 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:57,752 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2151 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:04:57,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:04:57,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:04:57,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:04:57,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:04:58,044 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:04:59,045 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:04:59,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:04:59,771 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:04:59,772 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:04:59,772 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4171 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:04:59,773 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:04:59,773 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:04:59,773 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:04:59,773 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:00,045 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:00,162 WARN [HBase-Metrics2-1 {}] impl.MetricsConfig(136): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-10-15T16:05:01,045 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:01,087 DEBUG [HBase-Metrics2-1 {}] impl.GlobalMetricRegistriesAdapter(153): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-10-15T16:05:01,087 DEBUG [HBase-Metrics2-1 {}] impl.GlobalMetricRegistriesAdapter(153): Removing adapter for the MetricRegistry: RegionServer,sub=TableRequests_Namespace_hbase_table_meta 2023-10-15T16:05:01,087 DEBUG [HBase-Metrics2-1 {}] impl.GlobalMetricRegistriesAdapter(153): Removing adapter for the MetricRegistry: RegionServer,sub=TableRequests_Namespace_default_table_SyncRep 2023-10-15T16:05:01,087 DEBUG [HBase-Metrics2-1 {}] impl.GlobalMetricRegistriesAdapter(153): Removing adapter for the MetricRegistry: RegionServer,sub=TableRequests_Namespace_hbase_table_replication 2023-10-15T16:05:02,046 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:02,227 INFO [master/8c5d72df62de:0.Chore.1 {}] migrate.RollingUpgradeChore(116): There is no table to migrate StoreFileTracker! 2023-10-15T16:05:02,227 INFO [master/8c5d72df62de:0.Chore.1 {}] migrate.RollingUpgradeChore(85): All Rolling-Upgrade tasks are complete, shutdown RollingUpgradeChore! 2023-10-15T16:05:03,046 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:03,820 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:03,820 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:03,821 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:05:03,821 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8220 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:05:03,822 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:03,822 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:05:03,822 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:03,822 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:04,046 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:05,046 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:06,047 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:06,589 WARN [HBase-Metrics2-1 {}] impl.MetricsConfig(136): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-10-15T16:05:07,047 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:08,047 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:09,047 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:10,048 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:10,332 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 14.8410 sec 2023-10-15T16:05:11,048 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:12,048 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:13,048 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:13,830 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:13,830 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:13,831 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:05:13,832 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18231 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:05:13,832 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:13,832 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:05:13,832 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:13,832 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:14,049 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:15,049 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:15,332 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 19.8410 sec 2023-10-15T16:05:16,049 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:17,049 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:18,050 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:18,751 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:05:19,050 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:20,050 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:20,333 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 24.8420 sec 2023-10-15T16:05:21,050 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:22,051 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:23,051 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:23,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:23,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:23,891 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:05:23,892 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28291 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:05:23,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:23,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:05:23,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:23,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:24,051 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:25,051 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:25,333 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 29.8420 sec 2023-10-15T16:05:26,052 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:27,052 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:28,052 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:29,052 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:30,053 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:30,333 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 34.8420 sec 2023-10-15T16:05:31,053 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:32,053 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:33,053 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:33,970 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:33,971 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:33,971 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:05:33,972 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38371 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:05:33,972 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:33,972 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:05:33,972 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:33,973 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:34,053 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:35,054 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:35,334 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 39.8420 sec 2023-10-15T16:05:36,054 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:37,054 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:38,055 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:39,055 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:40,055 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:40,334 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 44.8430 sec 2023-10-15T16:05:41,055 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:42,056 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:43,056 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:44,030 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:44,030 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:44,031 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:05:44,032 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48431 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:05:44,032 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:44,032 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:05:44,032 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:44,032 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:44,056 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:45,056 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:45,334 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 49.8430 sec 2023-10-15T16:05:46,057 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:47,057 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:48,057 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:48,751 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:05:49,057 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:50,058 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:50,335 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 54.8430 sec 2023-10-15T16:05:51,058 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:52,059 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:53,059 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:54,059 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:55,060 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:55,335 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 59.8440 sec 2023-10-15T16:05:55,502 WARN [PEWorker-4 {}] replication.AssignReplicationQueuesProcedure(189): Failed to claim replication queues for 8c5d72df62de,46527,1697385859222, suspend 1 secs org.apache.hadoop.hbase.replication.ReplicationException: failed to listAllQueueIds, serverName=8c5d72df62de,46527,1697385859222 at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:256) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=2, exceptions: 2023-10-15T16:04:55.495Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:55.501Z, org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59893ms) waiting for region location for hbase:replication, row='', replicaId=0 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:245) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.tryScheduleRetry(AsyncRpcRetryingCaller.java:130) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.onError(AsyncRpcRetryingCaller.java:215) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:112) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59893ms) waiting for region location for hbase:replication, row='', replicaId=0 at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:05:55,503 INFO [PEWorker-4 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=63, ppid=56, state=WAITING_TIMEOUT:ASSIGN_REPLICATION_QUEUES_CLAIM, hasLock=true; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure; timeout=1000, timestamp=1697385956503 2023-10-15T16:05:55,600 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:55,600 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:55,601 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:05:55,601 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 13, maxAttempts = 46, timeout = 60000 ms, time elapsed = 60001 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:05:55,602 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:55,602 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:05:55,602 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:55,602 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncNonMetaRegionLocator(265): Failed to locate region in 'hbase:replication', row='', locateType=CURRENT org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:04:55.601Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:55.711Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:55.921Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:56.230Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:56.741Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:04:57.752Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:04:59.773Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:03.822Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:13.832Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:23.892Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:33.972Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:44.032Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:55.602Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.tryScheduleRetry(AsyncRpcRetryingCaller.java:130) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.onError(AsyncRpcRetryingCaller.java:215) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$call$4(AsyncSingleRequestRpcRetryingCaller.java:86) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncClientScanner.lambda$callOpenScanner$0(AsyncClientScanner.java:181) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.com.google.protobuf.RpcUtil$1.run(RpcUtil.java:79) ~[hbase-shaded-protobuf-4.1.5.jar:4.1.5] at org.apache.hbase.thirdparty.com.google.protobuf.RpcUtil$1.run(RpcUtil.java:70) ~[hbase-shaded-protobuf-4.1.5.jar:4.1.5] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:396) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 34 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:05:55,604 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:56,060 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:56,506 DEBUG [PEWorker-5 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:56,506 DEBUG [PEWorker-5 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:56,507 DEBUG [PEWorker-5 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:05:56,507 DEBUG [PEWorker-5 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:56,507 DEBUG [PEWorker-5 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:56,507 DEBUG [PEWorker-5 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:56,610 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:56,610 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:56,610 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:05:56,610 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:56,610 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:56,610 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:56,820 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:56,820 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:56,820 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:05:56,821 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:56,821 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:56,821 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:57,060 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:57,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:57,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:57,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:05:57,131 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:57,131 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:57,131 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:57,640 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:57,641 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:57,641 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:05:57,642 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:57,642 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:05:57,642 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:57,642 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:58,060 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:05:58,650 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:05:58,650 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:05:58,650 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:05:58,651 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2144 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:05:58,651 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:58,651 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:58,651 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:05:59,060 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:00,061 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:00,670 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:06:00,670 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:06:00,671 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:06:00,672 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4165 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:06:00,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:00,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:06:00,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:06:00,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:01,061 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:02,061 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:03,061 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:04,062 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:04,680 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:06:04,680 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:06:04,681 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:06:04,685 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8178 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:06:04,685 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:04,685 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:06:04,685 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:06:04,686 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:05,062 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:06,062 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:07,063 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:08,063 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:09,063 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:10,063 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:10,335 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-5(pid=63), run time 13.8310 sec 2023-10-15T16:06:11,064 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:12,064 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:13,064 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:14,064 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:14,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:06:14,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:06:14,751 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:06:14,752 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18245 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:06:14,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:14,753 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:06:14,753 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:06:14,753 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:15,065 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:15,336 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-5(pid=63), run time 18.8320 sec 2023-10-15T16:06:16,065 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:17,065 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:18,065 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:18,751 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:06:19,066 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:20,066 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:20,336 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-5(pid=63), run time 23.8320 sec 2023-10-15T16:06:21,066 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:21,599 INFO [ProcExecTimeout {}] assignment.AssignmentManager$DeadServerMetricRegionChore(1502): Found 3 OPEN regions on dead servers and 0 OPEN regions on unknown servers 2023-10-15T16:06:22,066 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:23,066 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:24,067 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:24,840 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:06:24,840 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:06:24,841 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:06:24,842 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28335 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:06:24,842 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:24,842 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:06:24,842 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:06:24,842 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:25,067 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:25,336 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-5(pid=63), run time 28.8320 sec 2023-10-15T16:06:26,067 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:27,067 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:28,068 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:29,068 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:30,068 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:30,337 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-5(pid=63), run time 33.8320 sec 2023-10-15T16:06:31,069 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:32,069 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:33,069 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:34,069 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:34,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:06:34,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:06:34,891 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:06:34,891 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38385 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:06:34,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:34,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:06:34,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:06:34,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:35,070 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:35,337 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-5(pid=63), run time 38.8330 sec 2023-10-15T16:06:36,070 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:37,070 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:38,070 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:39,071 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:40,071 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:40,337 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-5(pid=63), run time 43.8330 sec 2023-10-15T16:06:41,071 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:42,071 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:43,072 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:44,072 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:44,940 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:06:44,940 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:06:44,941 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:06:44,942 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48435 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:06:44,942 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:44,942 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:06:44,942 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:06:44,942 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:45,072 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:45,338 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-5(pid=63), run time 48.8330 sec 2023-10-15T16:06:46,072 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:47,073 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:48,073 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:48,751 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:06:49,073 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:50,073 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:50,338 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-5(pid=63), run time 53.8340 sec 2023-10-15T16:06:51,074 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:52,074 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:53,074 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:54,074 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:55,075 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:55,338 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-5(pid=63), run time 58.8340 sec 2023-10-15T16:06:56,075 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:56,510 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncNonMetaRegionLocator(265): Failed to locate region in 'hbase:replication', row='', locateType=CURRENT org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:05:56.507Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:56.610Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:56.821Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:57.131Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:05:57.642Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:05:58.651Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:00.672Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:04.685Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:14.753Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:24.842Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:34.892Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:44.942Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:06:56,510 WARN [PEWorker-5 {}] replication.AssignReplicationQueuesProcedure(189): Failed to claim replication queues for 8c5d72df62de,46527,1697385859222, suspend 2 secs org.apache.hadoop.hbase.replication.ReplicationException: failed to listAllQueueIds, serverName=8c5d72df62de,46527,1697385859222 at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:256) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=1, exceptions: 2023-10-15T16:06:56.510Z, org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:245) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.tryScheduleRetry(AsyncRpcRetryingCaller.java:130) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.onError(AsyncRpcRetryingCaller.java:215) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:112) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:06:56,512 INFO [PEWorker-5 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=63, ppid=56, state=WAITING_TIMEOUT:ASSIGN_REPLICATION_QUEUES_CLAIM, hasLock=true; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure; timeout=2000, timestamp=1697386018512 2023-10-15T16:06:57,075 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:58,075 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:58,515 DEBUG [PEWorker-2 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:06:58,515 DEBUG [PEWorker-2 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:06:58,516 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:06:58,516 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:58,517 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:06:58,517 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:06:58,517 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:58,620 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:06:58,620 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:06:58,620 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:06:58,620 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:58,620 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:58,620 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:06:58,830 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:06:58,830 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:06:58,831 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:06:58,831 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:58,831 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:58,831 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:06:59,076 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:06:59,140 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:06:59,140 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:06:59,140 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:06:59,141 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:59,141 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:59,141 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:06:59,650 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:06:59,650 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:06:59,650 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:06:59,651 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:59,651 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:59,651 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:07:00,076 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:00,660 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:07:00,660 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:07:00,661 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:07:00,661 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2146 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:07:00,662 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:00,662 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:07:00,662 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:07:00,662 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:01,076 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:02,076 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:02,670 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:07:02,670 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:07:02,672 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:07:02,672 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4157 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:07:02,673 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:02,673 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:07:02,673 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:07:02,673 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:03,077 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:04,077 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:05,077 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:06,077 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:06,680 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:07:06,681 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:07:06,682 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:07:06,682 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8167 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:07:06,683 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:06,683 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:07:06,683 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:07:06,683 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:07,078 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:08,078 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:09,078 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:10,078 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:10,339 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-2(pid=63), run time 11.8270 sec 2023-10-15T16:07:11,079 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:12,079 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:13,079 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:14,079 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:15,080 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:15,339 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-2(pid=63), run time 16.8270 sec 2023-10-15T16:07:16,080 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:16,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:07:16,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:07:16,721 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:07:16,722 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18206 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:07:16,722 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:16,722 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:07:16,722 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:07:16,723 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:17,080 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:18,080 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:18,752 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:07:19,081 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:20,081 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:20,339 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-2(pid=63), run time 21.8270 sec 2023-10-15T16:07:21,081 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:22,081 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:23,082 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:24,082 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:25,082 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:25,340 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-2(pid=63), run time 26.8270 sec 2023-10-15T16:07:26,082 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:26,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:07:26,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:07:26,761 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:07:26,761 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28246 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:07:26,764 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:26,764 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:07:26,764 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:07:26,764 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:27,083 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:28,083 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:29,083 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:30,083 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:30,340 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-2(pid=63), run time 31.8280 sec 2023-10-15T16:07:31,084 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:32,084 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:33,084 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:34,084 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:35,085 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:35,340 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-2(pid=63), run time 36.8280 sec 2023-10-15T16:07:36,085 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:36,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:07:36,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:07:36,791 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:07:36,792 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38276 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:07:36,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:36,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:07:36,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:07:36,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:37,085 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:38,086 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:39,086 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:40,086 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:40,340 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-2(pid=63), run time 41.8280 sec 2023-10-15T16:07:41,086 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:42,087 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:43,087 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:44,087 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:45,087 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:45,341 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-2(pid=63), run time 46.8290 sec 2023-10-15T16:07:46,088 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:46,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:07:46,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:07:46,882 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:07:46,882 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48367 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:07:46,883 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:46,883 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:07:46,883 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:07:46,883 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:47,088 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:48,088 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:48,752 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:07:49,088 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:50,089 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:50,341 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-2(pid=63), run time 51.8290 sec 2023-10-15T16:07:51,089 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:52,089 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:53,089 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:54,090 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:55,090 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:55,342 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-2(pid=63), run time 56.8290 sec 2023-10-15T16:07:56,090 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:57,090 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:58,091 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:07:58,521 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncNonMetaRegionLocator(265): Failed to locate region in 'hbase:replication', row='', locateType=CURRENT org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:06:58.517Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:06:58.620Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:58.831Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:59.141Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:06:59.651Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:07:00.662Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:02.673Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:06.683Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:16.722Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:26.764Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:36.792Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:07:46.883Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:07:58,521 WARN [PEWorker-2 {}] replication.AssignReplicationQueuesProcedure(189): Failed to claim replication queues for 8c5d72df62de,46527,1697385859222, suspend 4 secs org.apache.hadoop.hbase.replication.ReplicationException: failed to listAllQueueIds, serverName=8c5d72df62de,46527,1697385859222 at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:256) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=1, exceptions: 2023-10-15T16:07:58.520Z, org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:245) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.tryScheduleRetry(AsyncRpcRetryingCaller.java:130) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.onError(AsyncRpcRetryingCaller.java:215) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:112) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:07:58,523 INFO [PEWorker-2 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=63, ppid=56, state=WAITING_TIMEOUT:ASSIGN_REPLICATION_QUEUES_CLAIM, hasLock=true; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure; timeout=4000, timestamp=1697386082522 2023-10-15T16:07:59,091 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:00,091 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:01,092 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:02,092 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:02,525 DEBUG [PEWorker-1 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:02,526 DEBUG [PEWorker-1 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:02,527 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:08:02,527 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:02,527 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:08:02,527 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:02,528 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:02,630 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:02,630 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:02,630 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:08:02,631 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:02,631 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:02,631 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:02,840 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:02,840 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:02,841 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:08:02,841 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:02,841 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:02,841 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:03,092 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:03,150 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:03,150 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:03,150 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:08:03,151 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:03,151 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:03,151 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:03,662 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:03,662 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:03,662 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:08:03,662 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:03,663 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:03,663 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:04,092 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:04,670 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:04,670 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:04,671 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:08:04,671 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2145 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:08:04,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:04,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:08:04,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:04,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:05,093 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:06,093 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:06,680 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:06,680 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:06,681 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:08:06,681 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4155 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:08:06,682 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:06,682 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:08:06,682 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:06,682 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:07,093 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:08,093 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:09,096 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:10,096 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:10,690 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:10,690 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:10,691 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:08:10,691 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8165 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:08:10,692 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:10,692 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:08:10,692 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:10,692 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:11,096 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:12,096 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:13,097 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:14,097 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:15,097 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:15,342 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 12.8200 sec 2023-10-15T16:08:16,097 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:17,098 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:18,098 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:18,752 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:08:19,098 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:20,099 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:20,342 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 17.8200 sec 2023-10-15T16:08:20,780 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:20,780 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:20,781 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:08:20,782 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18256 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:08:20,782 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:20,782 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:08:20,782 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:20,782 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:21,099 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:21,600 INFO [ProcExecTimeout {}] assignment.AssignmentManager$DeadServerMetricRegionChore(1502): Found 3 OPEN regions on dead servers and 0 OPEN regions on unknown servers 2023-10-15T16:08:22,099 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:23,099 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:24,100 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:25,100 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:25,343 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 22.8210 sec 2023-10-15T16:08:26,100 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:27,100 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:28,100 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:29,101 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:30,101 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:30,343 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 27.8210 sec 2023-10-15T16:08:30,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:30,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:30,811 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:08:30,812 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28286 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:08:30,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:30,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:08:30,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:30,813 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:31,101 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:32,102 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:33,102 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:34,102 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:35,103 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:35,343 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 32.8210 sec 2023-10-15T16:08:36,103 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:37,103 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:38,104 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:39,104 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:40,104 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:40,344 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 37.8210 sec 2023-10-15T16:08:40,830 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:40,830 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:40,831 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:08:40,831 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38305 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:08:40,832 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:40,832 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:08:40,832 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:40,832 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:41,104 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:42,105 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:43,105 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:44,105 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:45,105 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:45,344 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 42.8220 sec 2023-10-15T16:08:46,106 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:47,106 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:48,106 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:48,752 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:08:49,106 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:50,107 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:50,344 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 47.8220 sec 2023-10-15T16:08:50,870 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:08:50,870 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:08:50,872 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:08:50,875 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48349 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:08:50,875 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:50,875 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:08:50,875 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:08:50,876 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:51,107 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:52,107 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:53,107 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:54,108 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:55,108 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:55,345 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 52.8230 sec 2023-10-15T16:08:56,108 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:57,108 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:58,109 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:08:59,109 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:00,109 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:00,345 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 57.8230 sec 2023-10-15T16:09:01,109 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:02,110 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:02,530 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncNonMetaRegionLocator(265): Failed to locate region in 'hbase:replication', row='', locateType=CURRENT org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:08:02.527Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:02.631Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:02.841Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:03.151Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:03.663Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:08:04.672Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:06.682Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:10.692Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:20.782Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:30.812Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:40.832Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:08:50.875Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:09:02,534 WARN [PEWorker-1 {}] replication.AssignReplicationQueuesProcedure(189): Failed to claim replication queues for 8c5d72df62de,46527,1697385859222, suspend 8 secs org.apache.hadoop.hbase.replication.ReplicationException: failed to listAllQueueIds, serverName=8c5d72df62de,46527,1697385859222 at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:256) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=1, exceptions: 2023-10-15T16:09:02.530Z, org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:245) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.tryScheduleRetry(AsyncRpcRetryingCaller.java:130) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.onError(AsyncRpcRetryingCaller.java:215) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:112) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:02,536 INFO [PEWorker-1 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=63, ppid=56, state=WAITING_TIMEOUT:ASSIGN_REPLICATION_QUEUES_CLAIM, hasLock=true; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure; timeout=8000, timestamp=1697386150536 2023-10-15T16:09:03,110 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:04,110 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:05,110 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:06,110 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:07,111 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:08,111 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:09,111 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:10,111 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:10,549 DEBUG [PEWorker-3 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:10,549 DEBUG [PEWorker-3 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:10,550 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:10,550 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:10,551 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:09:10,551 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:10,551 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:10,659 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:10,660 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:10,660 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:10,660 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:10,660 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:10,660 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:10,870 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:10,870 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:10,870 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:10,870 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:10,870 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:10,870 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:11,112 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:11,180 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:11,180 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:11,181 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:11,181 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:11,181 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:11,181 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:11,690 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:11,690 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:11,690 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:11,690 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:11,690 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:11,690 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:12,077 DEBUG [master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics {}] regionserver.ChunkCreator$MemStoreChunkPool$StatisticsThread(417): data stats (chunk size=2097152): current pool size=7, created chunk count=19, reused chunk count=14, reuseRatio=42.42% 2023-10-15T16:09:12,092 DEBUG [master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics {}] regionserver.ChunkCreator$MemStoreChunkPool$StatisticsThread(417): index stats (chunk size=209715): current pool size=0, created chunk count=0, reused chunk count=0, reuseRatio=0 2023-10-15T16:09:12,112 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:12,700 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:12,700 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:12,701 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:12,701 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2152 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:09:12,702 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:12,702 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:09:12,702 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:12,702 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:13,112 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:14,112 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:14,710 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:14,710 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:14,711 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:14,711 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4162 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:09:14,712 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:14,712 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:09:14,712 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:14,712 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:15,112 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:16,113 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:17,113 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:18,113 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:18,740 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:18,740 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:18,741 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:18,741 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8192 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:09:18,741 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:18,741 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:09:18,741 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:18,742 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:18,752 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:09:19,113 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:20,114 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:20,996 WARN [HBase-Metrics2-1 {}] impl.MetricsConfig(136): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-10-15T16:09:21,114 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:21,604 INFO [master/8c5d72df62de:0.Chore.1 {}] master.HMaster(2059): Not running balancer because processing dead regionserver(s): 8c5d72df62de,38435,1697385859175, 8c5d72df62de,46527,1697385859222, 8c5d72df62de,35419,1697385888915, 8c5d72df62de,45999,1697385859285 2023-10-15T16:09:21,606 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(188): Hosts are {8c5d72df62de=0} racks are {/default-rack=0} 2023-10-15T16:09:21,606 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(287): server 0 has 3 regions 2023-10-15T16:09:21,606 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(294): server 0 is on host 0 2023-10-15T16:09:21,606 INFO [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(305): server 0 is on rack 0 2023-10-15T16:09:21,606 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(310): Number of tables=3, number of hosts=1, number of racks=1 2023-10-15T16:09:21,611 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:21,611 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:21,612 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:21,613 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:21,613 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:09:21,613 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:21,613 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:21,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:21,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:21,721 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:21,721 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:21,721 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:21,721 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:21,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:21,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:21,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:21,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:21,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:21,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:22,114 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:22,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:22,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:22,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:22,241 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:22,241 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:22,241 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:22,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:22,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:22,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:22,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:22,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:22,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:23,115 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:23,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:23,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:23,771 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:23,771 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2160 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:09:23,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:23,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:09:23,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:23,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:24,115 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:25,115 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:25,346 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 14.8100 sec 2023-10-15T16:09:25,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:25,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:25,791 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:25,791 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4180 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:09:25,791 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:25,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:09:25,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:25,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:26,115 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:27,115 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:28,116 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:28,820 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:28,820 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:28,821 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:28,821 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18272 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:09:28,822 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:28,822 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:09:28,822 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:28,822 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:29,116 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:29,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:29,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:29,811 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:29,811 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8200 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:09:29,812 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:29,812 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:29,812 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:30,116 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:30,346 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 19.8100 sec 2023-10-15T16:09:31,116 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:32,116 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:33,117 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:34,117 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:35,117 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:35,346 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 24.8100 sec 2023-10-15T16:09:36,117 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:37,118 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:38,118 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:38,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:38,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:38,921 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:38,922 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28373 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:09:38,922 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:38,922 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:09:38,922 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:38,923 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:39,118 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:39,840 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:39,840 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:39,840 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:39,840 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18229 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:09:39,841 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:39,841 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:39,841 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:40,118 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:40,346 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 29.8100 sec 2023-10-15T16:09:41,118 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:42,119 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:43,119 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:44,119 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:45,119 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:45,346 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 34.8100 sec 2023-10-15T16:09:46,120 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:47,120 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:48,120 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:48,753 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:09:49,000 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:49,000 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:49,001 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:49,002 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38453 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:09:49,002 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:49,002 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:09:49,002 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:49,003 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:49,120 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:49,850 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:49,850 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:49,850 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:49,851 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28240 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:09:49,852 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:49,852 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:49,852 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:50,121 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:50,346 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 39.8100 sec 2023-10-15T16:09:51,121 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:52,121 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:53,121 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:54,121 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:55,122 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:55,347 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 44.8110 sec 2023-10-15T16:09:56,122 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:57,122 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:58,122 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:59,090 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:59,090 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:59,091 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:09:59,092 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48543 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:09:59,092 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:59,092 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:09:59,092 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:09:59,093 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:59,123 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:09:59,860 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:09:59,860 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:09:59,860 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:09:59,861 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38249 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:09:59,862 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:59,862 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:59,862 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:00,123 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:00,347 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 49.8110 sec 2023-10-15T16:10:01,123 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:02,123 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:03,123 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:04,124 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:05,124 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:05,347 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 54.8110 sec 2023-10-15T16:10:06,124 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:07,124 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:08,125 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:09,125 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:09,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:09,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:09,891 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:09,891 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48280 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:10:09,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:09,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:09,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:09,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:10,125 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:10,348 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 59.8120 sec 2023-10-15T16:10:10,551 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncNonMetaRegionLocator(265): Failed to locate region in 'hbase:replication', row='', locateType=CURRENT org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:09:10.551Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:10.660Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:10.870Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:11.181Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:11.690Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:12.702Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:14.712Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:18.741Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:28.822Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:38.922Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:49.003Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:59.092Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:10:10,555 WARN [PEWorker-3 {}] replication.AssignReplicationQueuesProcedure(189): Failed to claim replication queues for 8c5d72df62de,46527,1697385859222, suspend 16 secs org.apache.hadoop.hbase.replication.ReplicationException: failed to listAllQueueIds, serverName=8c5d72df62de,46527,1697385859222 at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:256) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=1, exceptions: 2023-10-15T16:10:10.552Z, org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:09:10.551Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:10.660Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:10.870Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:11.181Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:11.690Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:12.702Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:14.712Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:18.741Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:28.822Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:38.922Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:49.003Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:59.092Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:245) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.tryScheduleRetry(AsyncRpcRetryingCaller.java:130) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.onError(AsyncRpcRetryingCaller.java:215) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:112) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$getRegionLocation$5(AsyncRegionLocator.java:167) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator$RegionLocationsFutureResult.complete(AsyncNonMetaRegionLocator.java:134) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.ArrayList.forEach(ArrayList.java:1541) ~[?:?] at org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator.complete(AsyncNonMetaRegionLocator.java:301) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator.access$500(AsyncNonMetaRegionLocator.java:68) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator$1.onError(AsyncNonMetaRegionLocator.java:416) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.lambda$openScanner$2(AsyncClientScanner.java:261) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:09:10.551Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:10.660Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:10.870Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:11.181Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:11.690Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:12.702Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:14.712Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:18.741Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:28.822Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:38.922Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:49.003Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:59.092Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:10,558 INFO [PEWorker-3 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=63, ppid=56, state=WAITING_TIMEOUT:ASSIGN_REPLICATION_QUEUES_CLAIM, hasLock=true; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure; timeout=16000, timestamp=1697386226558 2023-10-15T16:10:11,125 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:12,126 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:13,126 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:14,126 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:15,126 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:16,126 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:17,127 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:18,127 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:18,753 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:10:19,127 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:20,127 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:21,127 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:21,600 INFO [ProcExecTimeout {}] assignment.AssignmentManager$DeadServerMetricRegionChore(1502): Found 3 OPEN regions on dead servers and 0 OPEN regions on unknown servers 2023-10-15T16:10:21,620 ERROR [master/8c5d72df62de:0.Chore.1 {}] normalizer.RegionNormalizerChore(50): Failed to normalize regions. org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:09:21.613Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:21.721Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:21.930Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:22.241Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:22.751Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:23.772Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:25.792Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:09:29.812Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:39.841Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:49.852Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:09:59.862Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:09.892Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:493) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:461) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:422) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.fullScanTables(MetaTableAccessor.java:126) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.master.TableStateManager.getTablesInStates(TableStateManager.java:121) ~[classes/:?] at org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:2194) ~[classes/:?] at org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:48) ~[classes/:?] at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?] at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:10:21,622 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:21,622 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:21,623 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:21,623 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:21,623 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:21,623 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:21,624 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:21,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:21,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:21,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:10:21,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:21,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:21,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:21,940 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:21,940 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:21,940 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:10:21,941 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:21,941 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:21,941 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:22,128 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:22,250 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:22,250 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:22,250 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:10:22,251 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:22,251 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:22,251 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:22,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:22,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:22,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:10:22,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:22,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:22,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:23,128 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:23,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:23,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:23,771 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:23,772 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2149 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:10:23,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:23,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:23,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:23,773 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:24,128 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:25,128 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:25,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:25,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:25,791 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:25,791 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4169 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:10:25,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:25,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:25,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:25,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:26,128 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:26,561 DEBUG [PEWorker-4 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:26,562 DEBUG [PEWorker-4 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:26,562 DEBUG [PEWorker-4 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:10:26,562 DEBUG [PEWorker-4 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:26,562 DEBUG [PEWorker-4 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:26,562 DEBUG [PEWorker-4 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:26,670 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:26,670 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:26,671 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:10:26,671 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:26,671 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:26,671 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:26,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:26,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:26,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:10:26,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:26,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:26,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:27,129 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:27,190 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:27,190 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:27,190 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:10:27,190 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:27,190 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:27,190 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:27,700 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:27,700 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:27,701 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:10:27,701 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:27,701 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:27,701 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:28,129 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:28,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:28,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:28,720 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:28,721 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2159 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:10:28,723 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:28,723 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:28,723 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:28,724 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:29,129 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:29,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:29,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:29,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:10:29,811 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8188 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:10:29,811 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:29,811 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:29,811 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:30,129 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:30,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:30,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:30,731 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:30,732 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4170 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:10:30,732 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:30,732 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:30,732 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:30,733 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:31,130 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:32,130 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:33,130 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:34,130 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:34,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:34,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:34,751 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:34,751 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8189 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:10:34,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:34,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:34,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:34,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:35,130 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:36,131 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:37,131 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:38,131 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:39,132 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:39,850 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:39,850 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:39,851 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:39,852 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18229 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:10:39,852 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:39,852 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:39,852 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:39,853 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:40,132 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:40,349 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 13.7890 sec 2023-10-15T16:10:41,132 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:42,132 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:43,133 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:44,134 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:44,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:44,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:44,790 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:44,791 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18229 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:10:44,791 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:44,791 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:44,791 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:44,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:45,134 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:45,349 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 18.7900 sec 2023-10-15T16:10:46,134 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:47,134 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:48,134 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:48,753 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:10:49,135 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:49,910 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:49,911 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:49,911 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:49,914 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28291 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:10:49,914 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:49,914 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:49,914 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:49,914 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:50,135 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:50,349 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 23.7900 sec 2023-10-15T16:10:51,135 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:52,135 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:53,135 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:54,136 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:54,860 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:54,860 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:54,861 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:54,861 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28300 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:10:54,862 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:54,862 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:54,862 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:54,862 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:55,136 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:55,349 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 28.7900 sec 2023-10-15T16:10:56,136 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:57,136 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:58,137 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:59,137 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:10:59,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:10:59,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:10:59,921 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:10:59,922 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38299 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:10:59,922 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:59,922 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:10:59,922 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:10:59,922 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:00,137 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:00,350 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 33.7910 sec 2023-10-15T16:11:01,137 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:02,138 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:03,138 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:04,138 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:04,910 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:11:04,910 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:11:04,911 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:11:04,911 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38349 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:11:04,912 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:04,912 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:11:04,912 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:11:04,912 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:05,138 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:05,350 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 38.7910 sec 2023-10-15T16:11:06,138 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:07,139 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:08,139 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:09,139 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:10,030 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:11:10,030 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:11:10,031 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:11:10,031 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48409 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:11:10,032 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:10,032 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:11:10,032 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:11:10,032 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:10,139 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:10,350 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 43.7910 sec 2023-10-15T16:11:11,139 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:12,140 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:13,140 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:14,140 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:15,000 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:11:15,000 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:11:15,001 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:11:15,001 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48439 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:11:15,002 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:15,002 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:11:15,002 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:11:15,002 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:15,140 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:15,351 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 48.7910 sec 2023-10-15T16:11:16,140 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:17,141 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:18,141 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:18,753 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:11:19,141 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:20,141 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:20,351 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 53.7920 sec 2023-10-15T16:11:21,142 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:21,630 WARN [master/8c5d72df62de:0.Chore.1 {}] janitor.CatalogJanitor(144): Failed janitorial scan of hbase:meta table org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:10:21.623Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:21.730Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:21.941Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:22.251Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:22.760Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:23.772Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:25.792Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:29.811Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:39.852Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:49.914Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:59.922Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:10.032Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:493) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:461) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:415) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMetaForTableRegions(MetaTableAccessor.java:410) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.scanForReport(CatalogJanitor.java:236) ~[classes/:?] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.scan(CatalogJanitor.java:171) ~[classes/:?] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.chore(CatalogJanitor.java:136) ~[classes/:?] at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?] at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:11:21,632 INFO [master/8c5d72df62de:0.Chore.1 {}] hbase.ScheduledChore(145): Chore: 8c5d72df62de,33403,1697385859093-ClusterStatusChore missed its start time 2023-10-15T16:11:22,142 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:23,142 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:24,142 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:25,142 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:25,351 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-4(pid=63), run time 58.7920 sec 2023-10-15T16:11:26,143 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:26,571 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncNonMetaRegionLocator(265): Failed to locate region in 'hbase:replication', row='', locateType=CURRENT org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:10:26.562Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:26.671Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:26.880Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:27.190Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:27.701Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:10:28.723Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:30.732Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:34.752Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:44.791Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:10:54.862Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:04.912Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:15.002Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:11:26,571 WARN [PEWorker-4 {}] replication.AssignReplicationQueuesProcedure(189): Failed to claim replication queues for 8c5d72df62de,46527,1697385859222, suspend 32 secs org.apache.hadoop.hbase.replication.ReplicationException: failed to listAllQueueIds, serverName=8c5d72df62de,46527,1697385859222 at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:256) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=1, exceptions: 2023-10-15T16:11:26.570Z, org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:245) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.tryScheduleRetry(AsyncRpcRetryingCaller.java:130) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.onError(AsyncRpcRetryingCaller.java:215) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:112) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:11:26,572 INFO [PEWorker-4 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=63, ppid=56, state=WAITING_TIMEOUT:ASSIGN_REPLICATION_QUEUES_CLAIM, hasLock=true; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure; timeout=32000, timestamp=1697386318572 2023-10-15T16:11:27,143 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:28,143 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:29,143 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:30,143 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:31,144 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:32,144 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:33,144 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:34,144 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:35,144 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:36,145 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:37,145 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:38,145 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:39,145 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:40,145 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:41,146 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:42,146 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:43,146 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:44,146 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:45,147 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:46,147 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:47,147 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:48,147 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:48,753 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:11:49,147 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:50,148 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:51,148 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:52,148 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:53,148 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:54,148 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:55,149 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:56,149 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:57,149 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:58,149 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:58,574 DEBUG [PEWorker-1 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:11:58,574 DEBUG [PEWorker-1 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:11:58,575 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:11:58,575 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:58,575 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:11:58,575 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:11:58,576 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:58,680 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:11:58,680 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:11:58,680 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:11:58,680 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:11:58,680 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:11:58,681 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:11:58,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:11:58,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:11:58,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:11:58,891 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:11:58,891 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:11:58,891 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:11:59,149 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:11:59,200 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:11:59,200 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:11:59,200 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:11:59,201 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:11:59,201 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:11:59,201 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:11:59,710 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:11:59,710 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:11:59,711 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:11:59,711 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:11:59,711 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:11:59,711 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:12:00,150 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:00,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:12:00,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:12:00,721 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:12:00,721 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2147 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:12:00,721 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:00,721 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:12:00,721 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:12:00,722 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:01,150 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:02,150 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:02,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:12:02,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:12:02,731 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:12:02,731 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4157 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:12:02,732 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:02,732 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:12:02,732 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:12:02,732 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:03,150 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:04,150 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:05,151 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:06,151 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:06,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:12:06,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:12:06,771 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:12:06,772 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8197 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:12:06,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:06,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:12:06,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:12:06,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:07,151 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:08,151 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:09,151 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:10,152 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:10,352 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 11.7800 sec 2023-10-15T16:12:11,152 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:12,152 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:13,152 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:14,152 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:15,153 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:15,353 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 16.7800 sec 2023-10-15T16:12:16,153 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:16,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:12:16,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:12:16,791 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:12:16,791 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18217 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:12:16,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:16,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:12:16,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:12:16,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:17,153 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:18,153 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:18,754 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:12:19,154 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:20,154 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:20,353 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 21.7810 sec 2023-10-15T16:12:21,154 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:21,600 INFO [ProcExecTimeout {}] assignment.AssignmentManager$DeadServerMetricRegionChore(1502): Found 3 OPEN regions on dead servers and 0 OPEN regions on unknown servers 2023-10-15T16:12:22,154 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:23,154 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:24,155 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:25,155 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:25,353 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 26.7810 sec 2023-10-15T16:12:26,155 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:26,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:12:26,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:12:26,881 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:12:26,881 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28307 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:12:26,882 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:26,882 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:12:26,882 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:12:26,882 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:27,155 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:28,156 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:29,156 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:30,156 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:30,353 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 31.7810 sec 2023-10-15T16:12:31,156 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:32,156 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:33,157 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:34,157 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:35,157 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:35,354 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 36.7820 sec 2023-10-15T16:12:36,157 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:36,950 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:12:36,951 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:12:36,952 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:12:36,952 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38378 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:12:36,953 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:36,953 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:12:36,953 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:12:36,953 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:37,157 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:38,158 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:39,158 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:40,158 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:40,354 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 41.7820 sec 2023-10-15T16:12:41,158 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:42,159 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:43,159 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:44,159 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:45,159 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:45,354 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 46.7820 sec 2023-10-15T16:12:46,159 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:47,020 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:12:47,021 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:12:47,022 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:12:47,022 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48448 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:12:47,023 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:47,023 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:12:47,023 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:12:47,023 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:47,160 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:48,160 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:48,754 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:12:49,160 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:50,160 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:50,355 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 51.7820 sec 2023-10-15T16:12:51,161 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:52,161 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:53,161 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:54,162 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:55,162 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:55,355 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 56.7830 sec 2023-10-15T16:12:56,162 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:57,162 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:58,162 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:12:58,580 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncNonMetaRegionLocator(265): Failed to locate region in 'hbase:replication', row='', locateType=CURRENT org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:11:58.575Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:11:58.681Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:11:58.891Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:11:59.201Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:11:59.711Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:12:00.721Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:02.732Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:06.772Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:16.792Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:26.882Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:36.953Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:12:47.023Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:12:58,582 WARN [PEWorker-1 {}] replication.AssignReplicationQueuesProcedure(189): Failed to claim replication queues for 8c5d72df62de,46527,1697385859222, suspend 64 secs org.apache.hadoop.hbase.replication.ReplicationException: failed to listAllQueueIds, serverName=8c5d72df62de,46527,1697385859222 at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:256) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=1, exceptions: 2023-10-15T16:12:58.579Z, org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:245) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.tryScheduleRetry(AsyncRpcRetryingCaller.java:130) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.onError(AsyncRpcRetryingCaller.java:215) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:112) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:12:58,583 INFO [PEWorker-1 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=63, ppid=56, state=WAITING_TIMEOUT:ASSIGN_REPLICATION_QUEUES_CLAIM, hasLock=true; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure; timeout=64000, timestamp=1697386442583 2023-10-15T16:12:59,163 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:00,163 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:01,163 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:02,163 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:03,164 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:04,164 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:05,164 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:06,164 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:07,164 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:08,165 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:09,165 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:10,165 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:11,165 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:12,166 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:13,166 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:14,166 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:15,166 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:16,167 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:17,167 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:18,167 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:18,754 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:13:19,167 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:20,167 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:21,169 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:22,169 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:23,169 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:24,170 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:25,170 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:26,170 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:27,170 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:28,171 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:29,171 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:30,171 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:31,171 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:32,172 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:33,172 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:34,172 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:35,172 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:36,173 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:37,173 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:38,173 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:39,173 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:40,173 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:41,174 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:42,174 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:43,174 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:44,174 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:45,175 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:46,175 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:47,175 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:48,175 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:48,754 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:13:49,175 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:50,176 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:51,176 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:52,176 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:53,176 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:54,177 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:55,177 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:56,177 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:57,177 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:58,178 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:13:59,178 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:00,178 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:01,178 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:02,178 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:02,592 DEBUG [PEWorker-3 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:02,592 DEBUG [PEWorker-3 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:02,593 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:02,594 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:02,594 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:14:02,594 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:02,594 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:02,700 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:02,700 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:02,701 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:02,701 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:02,701 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:02,701 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:02,910 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:02,910 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:02,910 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:02,910 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:02,911 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:02,912 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:03,179 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:03,220 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:03,220 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:03,221 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:03,221 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:03,221 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:03,221 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:03,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:03,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:03,731 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:03,731 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:03,731 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:03,731 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:04,179 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:04,740 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:04,740 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:04,741 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:04,741 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2149 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:14:04,742 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:04,742 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:14:04,742 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:04,742 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:05,179 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:06,179 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:06,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:06,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:06,771 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:06,771 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4179 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:14:06,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:06,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:14:06,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:06,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:07,180 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:08,180 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:09,180 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:10,180 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:10,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:10,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:10,791 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:10,792 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8199 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:14:10,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:10,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:14:10,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:10,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:11,180 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:12,077 DEBUG [master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics {}] regionserver.ChunkCreator$MemStoreChunkPool$StatisticsThread(417): data stats (chunk size=2097152): current pool size=7, created chunk count=19, reused chunk count=14, reuseRatio=42.42% 2023-10-15T16:14:12,092 DEBUG [master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics {}] regionserver.ChunkCreator$MemStoreChunkPool$StatisticsThread(417): index stats (chunk size=209715): current pool size=0, created chunk count=0, reused chunk count=0, reuseRatio=0 2023-10-15T16:14:12,181 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:13,181 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:14,181 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:15,181 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:15,357 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 12.7730 sec 2023-10-15T16:14:16,182 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:16,420 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(83): >> Begin NameNode metrics dump 2023-10-15T16:14:16,488 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_num_ops=0 2023-10-15T16:14:16,488 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_mean=0 2023-10-15T16:14:16,488 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_num_ops=0 2023-10-15T16:14:16,488 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_25th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_95th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_90th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_90th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_max=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_98th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_25th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_75th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_min=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_75th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_min=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_max=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_max=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_25th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99.9th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_75th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_mean=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99.9th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_75th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_median=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_95th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_98th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99.9th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_median=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_25th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Context=master 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_median=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_95th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_98th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_90th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_mean=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99.9th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_95th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_min=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_max=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_98th_percentile=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_mean=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_num_ops=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_min=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_median=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_num_ops=0 2023-10-15T16:14:16,489 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_90th_percentile=0 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:HttpPort=null 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XceiverCount=3 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"2","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-15... 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:ClusterId=testClusterID 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test... 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DatanodeHostname=127.0.0.1 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SecurityEnabled=false 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:RpcPort=0 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DataPort=0 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SoftwareVersion=3.2.4 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SlowDisks=null 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XmitsInProgress=0 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DNStartedTimeInMillis=1697385856691 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Context=regionserver 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.port=44717 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Context=rpcdetailed 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthNumOps=50 2023-10-15T16:14:16,523 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpNumOps=4 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWrite=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosNumOps=44 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRemoved=7 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalAvgTime=0.9795918367346939 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsNumOps=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksCached=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsAvgTime=2.0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DataNodeActiveXceiversCount=1 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalNumOps=200 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsNumOps=53 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesRead=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpAvgTime=3792.75 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesAvgTime=0.0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionTasks=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncCount=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:NumProcessedCommands=201 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpNumOps=92 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromLocalClient=48 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:VolumeFailures=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosNumOps=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesRead=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosAvgTime=1433727.0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalReadTime=16 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesRead=298317 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsAvgTime=1.0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromRemoteClient=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksUncached=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DatanodeNetworkErrors=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromLocalClient=92 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksInPendingIBR=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockVerificationFailures=0 2023-10-15T16:14:16,524 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosNumOps=184 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesWrite=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosNumOps=184 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.SessionId=null 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsNumOps=1 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesWritten=973747 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRead=92 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReplicated=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Context=dfs 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksVerified=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromRemoteClient=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosAvgTime=20216.833333333332 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsNumOps=200 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksWritten=48 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsAvgTime=0.9183673469387755 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalWriteTime=21 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcDecodingTimeNanos=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesNumOps=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosNumOps=1700 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpNumOps=48 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosAvgTime=5276.454545454545 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesWritten=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosAvgTime=12045.5 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpNumOps=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,525 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumFailedVolumes=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheUsed=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheCapacity=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksCached=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:LastVolumeFailureDate=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Capacity=1054589059072 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Context=FSDatasetState 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:DfsUsed=625022 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Remaining=941225311222 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToCache=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToUnCache=7 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheCleared=1 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheHit=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Context=rpc 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheUpdated=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalDataFileIos=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalFileIoErrors=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateNumOps=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateNumOps=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateNumOps=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateNumOps=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateNumOps=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateNumOps=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Context=dfs 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalMetadataOperations=0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,526 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:MissingECBlockGroups=0 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:LowRedundancyECBlockGroups=0 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:BytesInFutureECBlockGroups=0 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:CorruptECBlockGroups=0 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:PendingDeletionECBlocks=0 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:TotalECBlockGroups=0 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheCleared=1 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheHit=0 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Context=rpc 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheUpdated=0 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:LastHATransitionTime=0 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:HostAndPort=localhost:39027 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:State=active 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowPeersReport=null 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowDisksReport=null 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:NNRole=NameNode 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:BytesWithFutureGenerationStamps=0 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SecurityEnabled=false 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.port=35753 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Context=rpcdetailed 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthNumOps=55 2023-10-15T16:14:16,527 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:ReceivedBytes=18480 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcSlowCalls=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationSuccesses=55 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumDroppedConnections=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcClientBackoff=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationFailures=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumOpenConnections=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.port=35753 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:SentBytes=11751 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeNumOps=55 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationSuccesses=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:CallQueueLength=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Context=rpc 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationFailures=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeNumOps=55 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeNumOps=55 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:ReceivedBytes=16800 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcSlowCalls=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationSuccesses=50 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumDroppedConnections=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcClientBackoff=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationFailures=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumOpenConnections=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.port=44717 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:SentBytes=10673 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeNumOps=50 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationSuccesses=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:CallQueueLength=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Context=rpc 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationFailures=0 2023-10-15T16:14:16,528 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeNumOps=50 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeNumOps=50 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemMaxM=2200.0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTerminated=0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsWaiting=86 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.SessionId=null 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Context=jvm 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapUsedM=188.82245 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTotalExtraSleepTime=51 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Old Generation=0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCount=11 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsBlocked=0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogWarn=0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsRunnable=83 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogInfo=0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsNew=0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapUsedM=323.47656 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTimedWaiting=119 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapMaxM=2200.0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Young Generation=11 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogError=0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.ProcessName=NameNode 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogFatal=0 2023-10-15T16:14:16,529 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillis=328 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumFailedVolumes=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheUsed=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheCapacity=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksCached=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:LastVolumeFailureDate=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Capacity=1054589059072 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Context=FSDatasetState 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:DfsUsed=382860 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Remaining=941225311222 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToCache=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToUnCache=7 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemMaxM=2200.0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTerminated=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsWaiting=86 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.SessionId=null 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Context=jvm 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapUsedM=188.82245 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTotalExtraSleepTime=22 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Old Generation=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCount=11 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsBlocked=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogWarn=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsRunnable=84 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogInfo=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsNew=0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapUsedM=323.47656 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTimedWaiting=118 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapMaxM=2200.0 2023-10-15T16:14:16,530 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Young Generation=11 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogError=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.ProcessName=DataNode 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogFatal=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillis=328 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_num_ops=1891 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99th_percentile=1 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_max=4 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_mean=206 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:appendCount=1891 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:lowReplicaRollRequest=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99th_percentile=332 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99.9th_percentile=4 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:sizeRollRequest=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendSize=297021 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_max=1 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowAppendCount=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:successfulLogRolls=24 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99.9th_percentile=333 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_TimeRangeCount_0-1=11 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_25th_percentile=36 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendSize=9557 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_90th_percentile=1 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendCount=1011 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_25th_percentile=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_10-100=4 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99th_percentile=4 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_98th_percentile=4 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_75th_percentile=298 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_75th_percentile=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_max=334 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_95th_percentile=4 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_mean=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendCount=48 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_100-1000=7 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:errorRollRequest=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:rollRequest=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_num_ops=1804 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_median=0 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_3-10=1 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_98th_percentile=1 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_min=36 2023-10-15T16:14:16,531 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_min=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_median=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Context=regionserver 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_0-1=10 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendCount=45 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_95th_percentile=326 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendSize=74104 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_median=262 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_98th_percentile=331 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_25th_percentile=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_75th_percentile=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_mean=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99.9th_percentile=1 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendSize=14494 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:writtenBytes=395176 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_90th_percentile=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_95th_percentile=1 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowSyncRollRequest=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_num_ops=1891 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_90th_percentile=319 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_min=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendCount=787 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicationOneBlocks=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:CorruptReplicatedBlocks=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicatedBlocks=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:TotalReplicatedBlocks=42 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalDataFileIos=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalFileIoErrors=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateNumOps=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateNumOps=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateNumOps=0 2023-10-15T16:14:16,532 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateNumOps=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateNumOps=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateNumOps=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Context=dfs 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalMetadataOperations=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotNumOps=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishNumOps=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotAvgTime=0.0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Context=metricssystem 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSources=67 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:DroppedPubAll=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishAvgTime=0.0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSources=67 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSinks=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSinks=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheCleared=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheHit=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Context=rpc 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheUpdated=134 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWrite=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosNumOps=28 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRemoved=0 2023-10-15T16:14:16,533 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsNumOps=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksCached=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsAvgTime=3.0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalNumOps=9 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesRead=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpAvgTime=9.0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpAvgTime=6810.7692307692305 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesAvgTime=0.0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionTasks=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncCount=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:NumProcessedCommands=10 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpNumOps=31 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromLocalClient=28 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:VolumeFailures=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosNumOps=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesRead=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosAvgTime=554484.75 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalReadTime=7 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesRead=57958 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsAvgTime=1.5714285714285714 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromRemoteClient=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksUncached=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DatanodeNetworkErrors=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromLocalClient=31 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksInPendingIBR=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockVerificationFailures=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosNumOps=62 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesWrite=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosNumOps=62 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.SessionId=null 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsNumOps=1 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesWritten=620489 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRead=31 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReplicated=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Context=dfs 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksVerified=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromRemoteClient=0 2023-10-15T16:14:16,534 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosAvgTime=19658.18181818182 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsNumOps=9 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksWritten=28 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsAvgTime=1.0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalWriteTime=12 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcDecodingTimeNanos=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesNumOps=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosNumOps=1285 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpNumOps=28 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosAvgTime=4301.205882352941 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesWritten=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosAvgTime=9971.454545454546 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpAvgTime=0.8181818181818182 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpNumOps=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemMaxM=2200.0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTerminated=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsWaiting=86 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.SessionId=null 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Context=jvm 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapUsedM=188.83234 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTotalExtraSleepTime=62 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Old Generation=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCount=11 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsBlocked=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogWarn=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsRunnable=83 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogInfo=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsNew=0 2023-10-15T16:14:16,535 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapUsedM=323.47656 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTimedWaiting=119 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapMaxM=2200.0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Young Generation=11 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogError=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.ProcessName=DataNode 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogFatal=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillis=328 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Context=regionserver 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:numTables=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99.9th_percentile=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_90th_percentile=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_95th_percentile=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterRequests=1101 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99th_percentile=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterWriteRequests=1059 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_mean=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterReadRequests=74 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_min=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_25th_percentile=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashSubmittedCount=4 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashFailedCount=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_98th_percentile=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Context=master 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_num_ops=2 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_75th_percentile=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_max=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_median=0 2023-10-15T16:14:16,536 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:receivedBytes=6048147 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationSuccesses=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_max=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFailures=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numOpenConnections=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_num_ops=1266 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_median=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationSuccesses=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_num_ops=1266 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_num_ops=1266 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_max=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_max=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_median=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.otherExceptions=3 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Context=master 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_median=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_min=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:14:16,537 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_median=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_max=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_min=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_min=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveHandler=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_num_ops=1266 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_mean=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReadHandler=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_median=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_num_ops=1266 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_mean=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callDropped=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions=3 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFallbacks=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationFailures=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_min=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_max=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:queueSize=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveScanHandler=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:14:16,538 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_min=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:sentBytes=16055 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumFailedVolumes=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheUsed=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheCapacity=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksCached=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:LastVolumeFailureDate=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Capacity=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Context=FSDatasetState 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:DfsUsed=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Remaining=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToCache=0 2023-10-15T16:14:16,539 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToUnCache=0 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:HttpPort=null 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XceiverCount=3 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"0","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-15... 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:ClusterId=testClusterID 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test... 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DatanodeHostname=127.0.0.1 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SecurityEnabled=false 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:RpcPort=0 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DataPort=0 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SoftwareVersion=3.2.4 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SlowDisks=null 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XmitsInProgress=0 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DNStartedTimeInMillis=1697385857417 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,540 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalDataFileIos=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalFileIoErrors=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateNumOps=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateNumOps=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateNumOps=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateNumOps=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateNumOps=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateNumOps=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Context=dfs 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalMetadataOperations=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumFailedVolumes=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheUsed=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheCapacity=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksCached=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:LastVolumeFailureDate=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Capacity=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Context=FSDatasetState 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:DfsUsed=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Remaining=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToCache=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToUnCache=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:ReceivedBytes=744936 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeAvgTime=0.009771986970684038 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcSlowCalls=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationSuccesses=27 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumDroppedConnections=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.NumOpenConnectionsPerUser={"jenkins":1} 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcClientBackoff=0 2023-10-15T16:14:16,541 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeAvgTime=0.006514657980456026 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationFailures=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumOpenConnections=1 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.port=39027 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:SentBytes=627674 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeNumOps=2342 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationSuccesses=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:CallQueueLength=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Context=rpc 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationFailures=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeNumOps=2342 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.serverName=ClientNamenodeProtocol 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeNumOps=2342 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.port=35085 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Context=rpcdetailed 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthNumOps=46 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemMaxM=2200.0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTerminated=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsWaiting=86 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.SessionId=null 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Context=jvm 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapUsedM=188.845 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTotalExtraSleepTime=30 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Old Generation=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCount=11 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsBlocked=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogWarn=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsRunnable=83 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogInfo=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsNew=0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapUsedM=323.47656 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTimedWaiting=119 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapMaxM=2200.0 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Young Generation=11 2023-10-15T16:14:16,542 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogError=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.ProcessName=DataNode 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogFatal=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillis=328 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsBatchedInSync=31 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsNumOps=224 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsAvgTime=0.13043478260869565 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportNumOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeNumOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteSnapshotOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsAvgTime=0.03896103896103896 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AllowSnapshotOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:RenameSnapshotOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeAvgTime=0.0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesCreated=80 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesAppended=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportAvgTime=0.0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageAvgTime=0.0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeNumOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeNumOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeNumOps=3 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeAvgTime=0.0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumTimesReReplicationNotScheduled=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SafeModeTime=3 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalNumOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetListingOps=66 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.ProcessName=NameNode 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageNumOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TotalFileOps=540 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSymlinkOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AddBlockOps=28 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DisallowSnapshotOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteFileOps=5 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsNumOps=255 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSnapshotOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PendingEditsCount=1 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ListSnapshottableDirOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockReceivedAndDeletedOps=96 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.SessionId=null 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditNumOps=0 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FsImageLoadTime=173 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateFileOps=36 2023-10-15T16:14:16,543 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Context=dfs 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesRenamed=17 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TimeoutReReplications=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsQueued=1 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedNumOps=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageNumOps=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageAvgTime=0.0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesInGetListingOps=35 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FileInfoOps=270 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetLinkTargetOps=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetAdditionalDatanodeOps=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsBatched=9 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesDeleted=1 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SuccessfulReReplications=2 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetBlockLocations=118 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesTruncated=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SnapshotDiffReportOps=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportNumOps=6 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeNumOps=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditAvgTime=0.0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportAvgTime=0.5 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumFailedVolumes=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheCapacity=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheUsed=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:FailedStorageLocations=[Ljava.lang.String;@19929b24 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksCached=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:LastVolumeFailureDate=0 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Capacity=1054589059072 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:14:16,544 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToUncache=7 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:DfsUsed=383910 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Remaining=941223623670 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToCache=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpNumOps=5 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWrite=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosNumOps=48 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRemoved=7 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalAvgTime=1.0303030303030303 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsNumOps=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksCached=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsAvgTime=2.0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DataNodeActiveXceiversCount=1 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalNumOps=200 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsNumOps=51 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesRead=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpAvgTime=3792.5 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesAvgTime=0.0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionTasks=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncCount=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:NumProcessedCommands=201 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpNumOps=99 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromLocalClient=48 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:VolumeFailures=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosNumOps=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesRead=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosAvgTime=388889.5 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalReadTime=29 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesRead=688752 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsAvgTime=0.75 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromRemoteClient=0 2023-10-15T16:14:16,545 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksUncached=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DatanodeNetworkErrors=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromLocalClient=99 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksInPendingIBR=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockVerificationFailures=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosNumOps=200 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesWrite=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosNumOps=200 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.SessionId=null 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsNumOps=1 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesWritten=973747 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRead=99 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReplicated=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Context=dfs 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksVerified=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromRemoteClient=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosAvgTime=45642.0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsNumOps=200 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksWritten=48 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsAvgTime=0.9797979797979798 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalWriteTime=22 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcDecodingTimeNanos=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesNumOps=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosNumOps=1700 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpNumOps=48 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosAvgTime=4914.0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesWritten=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosAvgTime=9294.0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpNumOps=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,546 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWrite=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosNumOps=30 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRemoved=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsNumOps=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksCached=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsAvgTime=95.0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalNumOps=9 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesRead=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpAvgTime=6809.0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesAvgTime=0.0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionTasks=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncCount=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:NumProcessedCommands=10 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpNumOps=35 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromLocalClient=28 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:VolumeFailures=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosNumOps=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesRead=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosAvgTime=352052.375 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalReadTime=16 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesRead=75811 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsAvgTime=1.3846153846153846 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromRemoteClient=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksUncached=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DatanodeNetworkErrors=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromLocalClient=35 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksInPendingIBR=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockVerificationFailures=0 2023-10-15T16:14:16,547 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosNumOps=70 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesWrite=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosNumOps=70 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.SessionId=null 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsNumOps=1 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesWritten=620489 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRead=35 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReplicated=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Context=dfs 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksVerified=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromRemoteClient=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosAvgTime=25843.916666666668 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsNumOps=9 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksWritten=28 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsAvgTime=1.0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalWriteTime=12 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcDecodingTimeNanos=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesNumOps=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosNumOps=1285 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpNumOps=28 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosAvgTime=4740.882352941177 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesWritten=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosAvgTime=9895.041666666666 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpAvgTime=0.75 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpNumOps=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,548 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpNumOps=4 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWrite=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosNumOps=32 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRemoved=7 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalAvgTime=0.9795918367346939 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsNumOps=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksCached=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsAvgTime=2.0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DataNodeActiveXceiversCount=1 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalNumOps=200 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsNumOps=55 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesRead=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpAvgTime=3793.25 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesAvgTime=0.0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionTasks=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncCount=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:NumProcessedCommands=201 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpNumOps=89 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromLocalClient=48 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:VolumeFailures=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosNumOps=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesRead=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosAvgTime=304427.5 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalReadTime=12 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesRead=339498 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsAvgTime=0.75 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromRemoteClient=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksUncached=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DatanodeNetworkErrors=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromLocalClient=89 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksInPendingIBR=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockVerificationFailures=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosNumOps=178 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesWrite=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosNumOps=178 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.SessionId=null 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsNumOps=1 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesWritten=973747 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,549 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRead=89 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReplicated=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Context=dfs 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksVerified=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromRemoteClient=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosAvgTime=17297.333333333332 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsNumOps=200 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksWritten=48 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsAvgTime=0.9489795918367347 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalWriteTime=28 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcDecodingTimeNanos=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesNumOps=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosNumOps=1700 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpNumOps=48 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosAvgTime=4986.545454545455 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesWritten=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosAvgTime=11669.166666666666 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpAvgTime=0.3333333333333333 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpNumOps=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDataNodeMessageCount=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastWrittenTransactionId=255 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:SnapshottableDirectories=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEncryptionZones=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReconstructionBlocks=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptBlocks=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastCheckpointTime=1697385845476 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ReadLockLongHoldCount=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumActiveClients=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastLogRoll=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumTimedOutPendingReconstructions=0 2023-10-15T16:14:16,550 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:VolumeFailuresTotal=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ProvidedCapacityTotal=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingECBlockGroups=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedGB=0.0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecommissioningDataNodes=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.HAState=active 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyECBlockGroups=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:FilesTotal=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ScheduledReplicationBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureECBlockGroups=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptECBlockGroups=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MillisSinceLastLoadedEdits=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalECBlockGroups=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExcessBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalLoad=32 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExpiredHeartbeats=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInServiceLiveDataNodes=3 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumStaleStorages=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:WriteLockLongHoldCount=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlocksTotal=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:UnderReplicatedBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicatedBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsed=495713 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalReplicatedBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:StaleDataNodes=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumFilesUnderConstruction=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemaining=2816043276636 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PostponedMisreplicatedBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedNonDFS=185085906847 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LockQueueLength=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptReplicatedBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Context=dfs 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomDeadDataNodes=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:Snapshots=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionECBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.TotalSyncTimes= 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReplicationBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotal=3163767177216 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDeadDataNodes=0 2023-10-15T16:14:16,551 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplOneBlocks=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalSyncCount=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastCheckpoint=255 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicationOneBlocks=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumLiveDataNodes=3 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlockCapacity=4194304 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotalGB=2946.0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyBlocks=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomLiveDataNodes=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemainingGB=2623.0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Context=regionserver 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheCleared=1 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheHit=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Context=rpc 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheUpdated=65 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemMaxM=2200.0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTerminated=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsWaiting=86 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.SessionId= 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Context=jvm 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapUsedM=188.84634 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Old Generation=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCount=11 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsBlocked=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogWarn=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsRunnable=83 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogInfo=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsNew=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapUsedM=324.47656 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTimedWaiting=119 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapMaxM=2200.0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Young Generation=11 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogError=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.ProcessName=Master 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogFatal=0 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillis=328 2023-10-15T16:14:16,552 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsCount=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeTotal=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointPercentComplete=1.0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageTotal=1 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsTotal=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointCount=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:ElapsedTime=681 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsPercentComplete=1.0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsElapsedTime=7 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointElapsedTime=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModePercentComplete=1.0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:PercentComplete=1.0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointTotal=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeCount=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeElapsedTime=13 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageElapsedTime=62 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageCount=1 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImagePercentComplete=1.0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumFailedVolumes=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheCapacity=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheUsed=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:FailedStorageLocations=[Ljava.lang.String;@78899ce1 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksCached=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:LastVolumeFailureDate=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Capacity=1054589059072 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToUncache=7 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:DfsUsed=382860 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Remaining=941223033846 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToCache=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsCount=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeTotal=0 2023-10-15T16:14:16,553 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointPercentComplete=1.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageTotal=1 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsTotal=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointCount=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:ElapsedTime=681 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsPercentComplete=1.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsElapsedTime=7 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointElapsedTime=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModePercentComplete=1.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:PercentComplete=1.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointTotal=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeCount=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeElapsedTime=13 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageElapsedTime=62 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageCount=1 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImagePercentComplete=1.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RackLocalityCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionCountSkewCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaHostCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ServerLocalityCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MemStoreSizeCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_95th_percentile=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_Overall=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ReadRequestCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_25th_percentile=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_98th_percentile=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_min=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Context=master 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_WriteRequestCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaRackCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99th_percentile=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_PrimaryRegionCountSkewCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:miscInvocationCount=18 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_median=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_num_ops=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_75th_percentile=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MoveCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_max=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_mean=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_CPRequestCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_TableSkewCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_StoreFileCostFunction=0.0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_90th_percentile=0 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.isBalancerActive=true 2023-10-15T16:14:16,554 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99.9th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_95th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99.9th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_median=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_mean=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_25th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_max=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_95th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_num_ops=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_25th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_98th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_min=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_mean=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_90th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_75th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_90th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_mean=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_num_ops=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_95th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_min=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_median=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_num_ops=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_mean=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_98th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_75th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_median=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99.9th_percentile=0 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Context=regionserver 2023-10-15T16:14:16,555 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_max=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_num_ops=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_num_ops=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_min=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_90th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_mean=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_95th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_max=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_75th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_75th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_min=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_num_ops=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_75th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_max=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_min=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_98th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_90th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_max=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_98th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_90th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_75th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_95th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_25th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_max=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_max=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_98th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_num_ops=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99.9th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_min=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_98th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_95th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_min=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_median=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:BlockBytesScannedCount=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99.9th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_25th_percentile=0 2023-10-15T16:14:16,556 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_median=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_90th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_95th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_num_ops=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_max=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_mean=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_max=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_min=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_mean=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_95th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99.9th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_25th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99.9th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_90th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_mean=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_95th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_min=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_75th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_max=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_max=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_90th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_max=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_98th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_25th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_median=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_median=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_75th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_mean=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99.9th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_median=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_90th_percentile=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:16,557 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_mean=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_mean=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_75th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_num_ops=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99.9th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_median=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_75th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_median=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_median=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_98th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_95th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_75th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_95th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_num_ops=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_max=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_98th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_25th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_95th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99.9th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_num_ops=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_25th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99.9th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_median=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_90th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_25th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_median=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_min=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_max=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_90th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_max=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_75th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_25th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_98th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_min=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_mean=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99.9th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_25th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_min=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_mean=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_min=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_98th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_min=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_98th_percentile=0 2023-10-15T16:14:16,558 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,559 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99.9th_percentile=0 2023-10-15T16:14:16,559 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_median=0 2023-10-15T16:14:16,559 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_num_ops=0 2023-10-15T16:14:16,559 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_median=0 2023-10-15T16:14:16,559 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_90th_percentile=0 2023-10-15T16:14:16,559 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,559 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_min=0 2023-10-15T16:14:16,559 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_25th_percentile=0 2023-10-15T16:14:16,559 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,559 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsCount=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_98th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_median=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestRatePerSecond=0.0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerMissCount=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_mean=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_median=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_98th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_95th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_median=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_90th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_5min_rate=0.0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_min=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_95th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitRatio=0.0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_75th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_90th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_num_ops=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_min=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_25th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestCount=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedOutputBytes=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_90th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkHitCount=1 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadOpsInCurThread=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_median=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestRatePerSecond=0.0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_25th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_min=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_num_ops=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_num_ops=26 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_90th_percentile=0 2023-10-15T16:14:16,560 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockBytesScannedCount=4389 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_max=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_75th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_max=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_mean=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_num_ops=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_25th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheEvictionCount=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99.9th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheCount=3 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:smallCompactionQueueLength=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCount=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_min=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_num_ops=15 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_mean=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowIncrementCount=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushQueueLength=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_mean=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_75th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:localBytesRead=108773 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushCount=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_median=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutateFailedCount=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_max=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorPoolAllocationBytes=1530880 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_98th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_min=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_median=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_min=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:updatesBlockedTime=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_median=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_mean=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitQueueLength=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_min=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_min=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_95th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCachingCount=3 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_25th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99.9th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileAge=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_median=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_75th_percentile=0 2023-10-15T16:14:16,561 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_max=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_95th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_num_ops=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_min=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cpRequestCount=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_90th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_median=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_max=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_num_ops=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALCount=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_98th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitRatio=0.7272727272727273 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataMissCount=1 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestCount=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissCount=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_75th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_mean=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheEvictionCount=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheFreeSize=922067336 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_95th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_median=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_75th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_max=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_median=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:activeScanners=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissCount=3 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_mean=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_max=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_max=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_95th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactionQueueLength=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_95th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_num_ops=0 2023-10-15T16:14:16,562 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_max=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_1min_rate=0.0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_min=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_median=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_num_ops=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_median=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_mean=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_75th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_min=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99.9th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_median=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_25th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_25th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_90th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_num_ops=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheSize=679544 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_num_ops=6 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_num_ops=26 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_98th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_count=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_median=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowGetCount=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_median=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterRequestsCount=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_min=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_min=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_25th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_max=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileSize=414534 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_min=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileCount=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99th_percentile=0 2023-10-15T16:14:16,563 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsCount=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99.9th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_98th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_num_ops=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedOutputBytes=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_max=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_num_ops=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_max=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticBloomSize=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexMissCount=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseInfoThresholdExceeded=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCount=3 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCount=3 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_max=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedToMob=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedMemstoreBytes=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheExpressHitPercent=72.72727272727273 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionServerStartTime=1697385859285 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedInputBytes=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedFromMob=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_max=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRequestCount=47 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_25th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcGetRequestCount=3 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_90th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeCount=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCountHitPercent=72.72727272727273 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_max=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_mean=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_min=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_98th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_min=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsSize=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_mean=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:16,564 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowDeleteCount=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockedRequestCount=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_mean=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheAccessCount=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_min=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_median=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_min=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_90th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_min=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_95th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_90th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheEvictedCount=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_90th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_max=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:avgStoreFileAge=1819 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_25th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheCount=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitRequestCount=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_num_ops=1 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_min=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalBytesRead=108773 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedToMob=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_max=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedInputBytes=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheSize=679544 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileCount=473 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_mean=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcScanRequestCount=26 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreOffHeapSize=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_75th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheMissCount=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowPutCount=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_max=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreSize=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_75th_percentile=0 2023-10-15T16:14:16,565 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsSize=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_mean=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_min=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_75th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_median=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_98th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_median=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Context=regionserver 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:scannerLeaseExpiredCount=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:minStoreFileAge=1782 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitCount=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_98th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcFullScanRequestCount=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_mean=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitSuccessCount=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_mean=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_75th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_25th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCountPrimary=3 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_max=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataBlockCount=1 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_98th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_max=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_median=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsCount=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_mean=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsCount=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCachingCount=8 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFailedInsertionCount=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_90th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsSize=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean_rate=0.0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticIndexSize=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorUsedBufferCount=2 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_min=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99.9th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissRatio=0.2727272727272727 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_75th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_max=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_mean=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,566 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_95th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMultiRequestCount=1 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheSize=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexHitCount=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_max=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_98th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedOutputBytes=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocal=0.0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_98th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_max=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_num_ops=3 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99.9th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_25th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_min=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_mean=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_15min_rate=0.0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99.9th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_min=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_95th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_75th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_median=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_median=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_mean=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_95th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitCount=8 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:largeCompactionQueueLength=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_mean=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutatePassedCount=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_max=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_95th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:16,567 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissRatio=0.0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsCount=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_min=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_mean=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_min=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:shortCircuitBytesRead=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99.9th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_median=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_min=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoMissCount=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileIndexSize=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99.9th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_mean=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerHitCount=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_98th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsSize=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCountPrimary=8 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_min=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_max=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99.9th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_90th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_90th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_num_ops=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_max=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_mean=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_max=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_min=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRowActionRequestCount=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMutateRequestCount=16 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheCount=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_25th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,568 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_mean=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataHitCount=7 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:zeroCopyBytesRead=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_98th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionCount=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_num_ops=1 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileCount=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_num_ops=1 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_median=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALSize=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_num_ops=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_mean=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_median=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheFreeSize=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:averageRegionSize=17474 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_95th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaMissCount=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_75th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:filteredReadRequestCount=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaHitCount=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_75th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_num_ops=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_95th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_median=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCount=8 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_num_ops=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.serverName=8c5d72df62de,45999,1697385859285 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReads=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowAppendCount=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:numReferenceFiles=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedFromMob=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSize=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_max=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_90th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_median=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_mean=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_98th_percentile=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadWins=0 2023-10-15T16:14:16,569 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseWarnThresholdExceeded=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_max=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_25th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoHitCount=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_median=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreHeapSize=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_25th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_min=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99.9th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_25th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_num_ops=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_90th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_max=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99.9th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_max=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_median=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsSize=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_median=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_median=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_min=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkMissCount=2 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_95th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFreeSize=922067336 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheHitPercent=0.0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemMaxM=2200.0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTerminated=0 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsWaiting=86 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.SessionId=null 2023-10-15T16:14:16,570 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Context=jvm 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapUsedM=188.85072 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTotalExtraSleepTime=30 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Old Generation=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCount=11 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsBlocked=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogWarn=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsRunnable=83 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogInfo=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsNew=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapUsedM=325.47656 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTimedWaiting=119 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapMaxM=2200.0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Young Generation=11 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogError=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.ProcessName=NameNode 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogFatal=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillis=328 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemMaxM=2200.0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTerminated=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsWaiting=86 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.SessionId=null 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Context=jvm 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapUsedM=188.85118 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTotalExtraSleepTime=40 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Old Generation=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCount=11 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsBlocked=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogWarn=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsRunnable=83 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogInfo=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsNew=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapUsedM=325.47656 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTimedWaiting=119 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapMaxM=2200.0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Young Generation=11 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogError=0 2023-10-15T16:14:16,571 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.ProcessName=DataNode 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogFatal=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillis=328 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_90th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_25th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_num_ops=14 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99.9th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_90th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_95th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_median=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_max=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_95th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_98th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_98th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_25th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:fsChecksumFailureCount=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_median=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_75th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_min=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Context=regionserver 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_max=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_min=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_90th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_75th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_max=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99.9th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_98th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_mean=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_95th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_num_ops=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_min=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_median=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99.9th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_mean=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_num_ops=186 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_25th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_mean=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_75th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99th_percentile=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context=dfs 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.1=0 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.2=dfs 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.2=8c5d72df62de 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.1=dfs 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.1=8c5d72df62de 2023-10-15T16:14:16,572 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.2=0 2023-10-15T16:14:16,573 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,573 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Context=master 2023-10-15T16:14:16,573 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:numMasterWALs=0 2023-10-15T16:14:16,573 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsCount=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_98th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_median=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestRatePerSecond=0.0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerMissCount=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_mean=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_median=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_98th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_95th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_median=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_90th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_5min_rate=0.0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_min=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_95th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitRatio=0.0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_75th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_90th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_num_ops=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_min=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_25th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestCount=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedOutputBytes=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_90th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkHitCount=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadOpsInCurThread=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_median=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestRatePerSecond=0.0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_25th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_min=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_num_ops=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_num_ops=2 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_90th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockBytesScannedCount=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_max=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_75th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_max=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,574 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_mean=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_num_ops=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_25th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheEvictionCount=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99.9th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheCount=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:smallCompactionQueueLength=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCount=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_min=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_num_ops=6 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_mean=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowIncrementCount=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushQueueLength=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_mean=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_75th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:localBytesRead=108773 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushCount=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_median=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutateFailedCount=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_max=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorPoolAllocationBytes=199680 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_98th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_min=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_median=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_min=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:updatesBlockedTime=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_median=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_mean=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitQueueLength=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_min=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_min=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_95th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCachingCount=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_25th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99.9th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileAge=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_median=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_75th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_max=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_95th_percentile=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_num_ops=1 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_min=0 2023-10-15T16:14:16,575 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cpRequestCount=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_90th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_median=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_max=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_num_ops=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALCount=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_98th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitRatio=0.0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataMissCount=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestCount=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissCount=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_75th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_mean=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheEvictionCount=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheFreeSize=922070024 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_95th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_median=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_75th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_num_ops=1 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_max=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_median=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:activeScanners=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissCount=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_mean=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_max=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_max=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_95th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactionQueueLength=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_95th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_num_ops=1 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_max=0 2023-10-15T16:14:16,576 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_1min_rate=0.0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_min=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_median=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_num_ops=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_median=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_mean=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_75th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_min=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99.9th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_median=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_25th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_25th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_90th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_num_ops=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheSize=676856 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_num_ops=2 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_98th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_count=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_median=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowGetCount=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_median=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterRequestsCount=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_min=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_min=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_25th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_max=0 2023-10-15T16:14:16,577 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileSize=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_min=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileCount=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsCount=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99.9th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_98th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_num_ops=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedOutputBytes=5860 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_max=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_num_ops=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_max=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticBloomSize=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexMissCount=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseInfoThresholdExceeded=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCount=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCount=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_max=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedToMob=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedMemstoreBytes=928 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheExpressHitPercent=0.0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionServerStartTime=1697385851561 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedInputBytes=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedFromMob=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_max=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRequestCount=11 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_25th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcGetRequestCount=3 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_90th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeCount=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCountHitPercent=0.0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_max=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_mean=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_min=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_98th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_min=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsSize=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_mean=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowDeleteCount=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockedRequestCount=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_mean=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheAccessCount=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_min=0 2023-10-15T16:14:16,578 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_median=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_min=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_90th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_min=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_95th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_90th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheEvictedCount=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_90th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_max=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:avgStoreFileAge=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_25th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheCount=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitRequestCount=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_min=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalBytesRead=108773 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedToMob=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_max=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedInputBytes=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheSize=676856 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileCount=239 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_mean=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcScanRequestCount=2 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreOffHeapSize=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_75th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheMissCount=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowPutCount=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_max=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreSize=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_75th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsSize=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_mean=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_min=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_75th_percentile=0 2023-10-15T16:14:16,579 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_median=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_98th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_median=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Context=regionserver 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:scannerLeaseExpiredCount=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:minStoreFileAge=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitCount=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_98th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcFullScanRequestCount=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_mean=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitSuccessCount=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_mean=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_75th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_25th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCountPrimary=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_max=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataBlockCount=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_98th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_max=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_median=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsCount=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_mean=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsCount=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCachingCount=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFailedInsertionCount=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_90th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsSize=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean_rate=0.0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticIndexSize=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_min=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99.9th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissRatio=0.0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_75th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_max=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_mean=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_95th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMultiRequestCount=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheSize=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,580 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexHitCount=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_max=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_98th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedOutputBytes=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocal=0.0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_98th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_max=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_num_ops=3 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99.9th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_25th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_min=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_mean=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_15min_rate=0.0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99.9th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_min=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_95th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_75th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_median=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_median=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_mean=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_95th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitCount=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:largeCompactionQueueLength=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_mean=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutatePassedCount=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_max=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_95th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissRatio=0.0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsCount=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_min=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_mean=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_min=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:shortCircuitBytesRead=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:16,581 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99.9th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_median=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_min=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoMissCount=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileIndexSize=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99.9th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_mean=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerHitCount=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_98th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsSize=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCountPrimary=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_min=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_max=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99.9th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_90th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_90th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_num_ops=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_max=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_mean=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_max=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_min=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRowActionRequestCount=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMutateRequestCount=6 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheCount=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_25th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_mean=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataHitCount=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:zeroCopyBytesRead=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_98th_percentile=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionCount=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_num_ops=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileCount=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_num_ops=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_median=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALSize=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_num_ops=0 2023-10-15T16:14:16,582 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_mean=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_median=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheFreeSize=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:averageRegionSize=928 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_95th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaMissCount=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_75th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:filteredReadRequestCount=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaHitCount=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_75th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_num_ops=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_95th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_median=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCount=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_num_ops=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.serverName=8c5d72df62de,34443,1697385851561 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReads=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowAppendCount=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:numReferenceFiles=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedFromMob=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSize=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_max=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_90th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_median=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_mean=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_98th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadWins=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseWarnThresholdExceeded=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_max=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_25th_percentile=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoHitCount=0 2023-10-15T16:14:16,583 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_median=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreHeapSize=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_25th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_min=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99.9th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_25th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_num_ops=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_90th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_max=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99.9th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_max=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_median=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsSize=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_median=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_median=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_min=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkMissCount=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_95th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFreeSize=922070024 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheHitPercent=0.0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:16,584 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsCount=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_98th_percentile=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_median=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestRatePerSecond=0.0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerMissCount=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_mean=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99th_percentile=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_median=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_98th_percentile=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_95th_percentile=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_median=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_90th_percentile=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_5min_rate=0.0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_min=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_95th_percentile=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitRatio=0.0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_75th_percentile=0 2023-10-15T16:14:16,585 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_90th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_num_ops=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_min=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_25th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestCount=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedOutputBytes=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_90th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkHitCount=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadOpsInCurThread=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_median=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestRatePerSecond=0.0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_25th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_min=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_num_ops=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_num_ops=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_90th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockBytesScannedCount=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_max=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_75th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_max=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_mean=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_num_ops=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_25th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheEvictionCount=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99.9th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheCount=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:smallCompactionQueueLength=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCount=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_min=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_num_ops=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_mean=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowIncrementCount=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushQueueLength=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_mean=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_75th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:localBytesRead=108773 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushCount=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_median=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutateFailedCount=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,586 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_max=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorPoolAllocationBytes=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_98th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_min=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_median=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_min=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:updatesBlockedTime=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_median=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_mean=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitQueueLength=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_min=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_min=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_95th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCachingCount=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_25th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99.9th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileAge=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_median=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_75th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_max=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_95th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_num_ops=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_min=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cpRequestCount=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_90th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_median=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_max=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_num_ops=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALCount=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_98th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitRatio=0.0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataMissCount=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestCount=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissCount=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_75th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_mean=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheEvictionCount=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheFreeSize=922070024 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_95th_percentile=0 2023-10-15T16:14:16,587 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_median=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_75th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_max=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_median=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:activeScanners=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissCount=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_mean=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_max=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_max=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_95th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactionQueueLength=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_95th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_num_ops=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_max=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_1min_rate=0.0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_min=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_median=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_num_ops=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_median=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_mean=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_75th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_min=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99.9th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_median=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_25th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_25th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_90th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_num_ops=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheSize=676856 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,588 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_num_ops=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_98th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_count=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_median=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowGetCount=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_median=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterRequestsCount=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_min=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_min=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_25th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_max=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileSize=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_min=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileCount=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsCount=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99.9th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_98th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_num_ops=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedOutputBytes=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_max=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_num_ops=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_max=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticBloomSize=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexMissCount=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseInfoThresholdExceeded=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCount=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCount=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_max=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedToMob=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedMemstoreBytes=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheExpressHitPercent=0.0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionServerStartTime=1697385851811 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedInputBytes=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedFromMob=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_max=0 2023-10-15T16:14:16,589 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRequestCount=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_25th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcGetRequestCount=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_90th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeCount=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCountHitPercent=0.0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_max=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_mean=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_min=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_98th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_min=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsSize=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_mean=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowDeleteCount=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockedRequestCount=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_mean=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheAccessCount=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_min=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_median=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_min=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_90th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_min=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_95th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_90th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheEvictedCount=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_90th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_max=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:avgStoreFileAge=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_25th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheCount=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitRequestCount=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_min=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalBytesRead=108773 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedToMob=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_max=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:16,590 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedInputBytes=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheSize=676856 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileCount=120 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_mean=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcScanRequestCount=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreOffHeapSize=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_75th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheMissCount=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowPutCount=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_max=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreSize=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_75th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsSize=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_mean=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_min=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_75th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_median=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_98th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_median=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Context=regionserver 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:scannerLeaseExpiredCount=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:minStoreFileAge=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitCount=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_98th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcFullScanRequestCount=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_mean=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitSuccessCount=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_mean=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_75th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_25th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCountPrimary=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_max=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataBlockCount=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_98th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_max=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_median=0 2023-10-15T16:14:16,591 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsCount=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_mean=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsCount=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCachingCount=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFailedInsertionCount=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_90th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsSize=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean_rate=0.0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticIndexSize=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorUsedBufferCount=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_min=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99.9th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissRatio=0.0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_75th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_max=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_mean=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_95th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMultiRequestCount=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheSize=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexHitCount=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_max=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_98th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedOutputBytes=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocal=0.0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_98th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_max=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_num_ops=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99.9th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_25th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_min=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_mean=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_15min_rate=0.0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99.9th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_min=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:16,592 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_95th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_75th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_median=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_median=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_mean=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_95th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitCount=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:largeCompactionQueueLength=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_mean=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutatePassedCount=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_max=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_95th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissRatio=0.0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsCount=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_min=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_mean=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_min=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:shortCircuitBytesRead=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99.9th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_median=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_min=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoMissCount=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileIndexSize=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99.9th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_mean=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerHitCount=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_98th_percentile=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsSize=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCountPrimary=0 2023-10-15T16:14:16,593 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_min=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_max=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99.9th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_90th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_90th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_num_ops=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_max=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_mean=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_max=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_min=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRowActionRequestCount=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMutateRequestCount=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheCount=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_25th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_mean=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataHitCount=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:zeroCopyBytesRead=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_98th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionCount=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_num_ops=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileCount=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_num_ops=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_median=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALSize=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_num_ops=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_mean=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_median=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheFreeSize=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:averageRegionSize=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_95th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaMissCount=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_75th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:filteredReadRequestCount=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaHitCount=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_75th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_num_ops=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_95th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_median=0 2023-10-15T16:14:16,594 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCount=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_num_ops=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.serverName=8c5d72df62de,40585,1697385851811 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReads=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowAppendCount=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:numReferenceFiles=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedFromMob=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSize=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_max=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_90th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_median=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_mean=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_98th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadWins=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseWarnThresholdExceeded=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_max=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_25th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoHitCount=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_median=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreHeapSize=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_25th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_min=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99.9th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_25th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_num_ops=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_90th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_max=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99.9th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_max=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_median=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsSize=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_median=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_median=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_min=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkMissCount=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,595 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:16,596 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_95th_percentile=0 2023-10-15T16:14:16,596 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFreeSize=922070024 2023-10-15T16:14:16,596 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,596 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheHitPercent=0.0 2023-10-15T16:14:16,596 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:16,596 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:16,596 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsCount=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_98th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_median=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestRatePerSecond=0.0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerMissCount=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_mean=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_median=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_98th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_95th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_median=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_90th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_5min_rate=0.0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_min=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_95th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitRatio=0.0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_75th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_90th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_num_ops=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_min=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_25th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestCount=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedOutputBytes=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_90th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkHitCount=1 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadOpsInCurThread=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_median=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestRatePerSecond=0.0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_25th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_min=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_num_ops=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_num_ops=11 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_90th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockBytesScannedCount=6840 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_max=0 2023-10-15T16:14:16,597 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_75th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_max=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_mean=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_num_ops=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_25th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheEvictionCount=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99.9th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheCount=4 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:smallCompactionQueueLength=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCount=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_min=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_num_ops=7 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_num_ops=1 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_mean=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowIncrementCount=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushQueueLength=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_mean=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_75th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:localBytesRead=108773 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushCount=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_median=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutateFailedCount=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_max=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorPoolAllocationBytes=599040 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_98th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_min=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_median=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_min=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:updatesBlockedTime=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_median=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_mean=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitQueueLength=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_min=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_min=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_95th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCachingCount=4 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_25th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99.9th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileAge=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_median=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_75th_percentile=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_max=0 2023-10-15T16:14:16,598 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_95th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_num_ops=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_min=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cpRequestCount=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_90th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_median=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_max=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_num_ops=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALCount=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_98th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitRatio=0.72 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataMissCount=6 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestCount=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissCount=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_75th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_mean=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheEvictionCount=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheFreeSize=922061632 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_95th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_median=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_75th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_max=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_median=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:activeScanners=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissCount=7 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_mean=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_max=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_max=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_95th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactionQueueLength=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_95th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,599 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_num_ops=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_max=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_1min_rate=0.0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_min=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_median=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_num_ops=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_median=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_mean=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_75th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_min=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99.9th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_median=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_25th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_25th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_90th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_num_ops=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheSize=685248 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_num_ops=7 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_num_ops=11 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_98th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_count=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_median=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowGetCount=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_median=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterRequestsCount=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_min=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_min=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_25th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_num_ops=1 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_max=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileSize=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_min=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileCount=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99th_percentile=0 2023-10-15T16:14:16,600 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsCount=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99.9th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_98th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_num_ops=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedOutputBytes=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_max=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_num_ops=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_max=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticBloomSize=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexMissCount=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseInfoThresholdExceeded=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCount=4 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCount=7 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_max=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedToMob=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedMemstoreBytes=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheExpressHitPercent=81.81818181818183 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionServerStartTime=1697385888915 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedInputBytes=18105 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedFromMob=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_max=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRequestCount=21 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_25th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcGetRequestCount=1 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_90th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeCount=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCountHitPercent=72.0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_max=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_mean=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_min=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_98th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_min=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsSize=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_mean=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowDeleteCount=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockedRequestCount=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_mean=0 2023-10-15T16:14:16,601 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheAccessCount=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_min=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_median=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_min=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_90th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_min=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_95th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_90th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheEvictedCount=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_90th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_max=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:avgStoreFileAge=12090 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_25th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheCount=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitRequestCount=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_num_ops=1 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_min=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalBytesRead=108773 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedToMob=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_max=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedInputBytes=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheSize=685248 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileCount=339 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_mean=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcScanRequestCount=11 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreOffHeapSize=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_75th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheMissCount=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowPutCount=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_max=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,602 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreSize=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_75th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsSize=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_mean=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_min=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_75th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_median=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_98th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_median=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Context=regionserver 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:scannerLeaseExpiredCount=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:minStoreFileAge=2959 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitCount=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_98th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcFullScanRequestCount=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_mean=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitSuccessCount=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_mean=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_75th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_25th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCountPrimary=7 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_max=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataBlockCount=3 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_98th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_max=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_median=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsCount=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_mean=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsCount=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCachingCount=18 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFailedInsertionCount=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_90th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsSize=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean_rate=0.0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticIndexSize=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_min=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99.9th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissRatio=0.28 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_75th_percentile=0 2023-10-15T16:14:16,603 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_max=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_mean=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_95th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMultiRequestCount=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheSize=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexHitCount=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_max=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_98th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedOutputBytes=5921 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocal=0.0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_98th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_max=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_num_ops=1 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99.9th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_25th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_min=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_mean=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_15min_rate=0.0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99.9th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_min=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_95th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_75th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_median=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_median=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_mean=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_95th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitCount=18 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:largeCompactionQueueLength=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_mean=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutatePassedCount=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_max=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_95th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissRatio=0.0 2023-10-15T16:14:16,604 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsCount=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_min=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_mean=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_min=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:shortCircuitBytesRead=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99.9th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_median=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_min=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoMissCount=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileIndexSize=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99.9th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_mean=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerHitCount=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_98th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsSize=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCountPrimary=18 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_min=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_max=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99.9th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_90th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_90th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_num_ops=1 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_max=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_mean=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_max=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_min=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRowActionRequestCount=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMutateRequestCount=8 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheCount=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_25th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationBytes=89913 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:16,605 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_mean=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataHitCount=17 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:zeroCopyBytesRead=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_98th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionCount=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_num_ops=1 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileCount=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_num_ops=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_median=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALSize=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_num_ops=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_mean=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_median=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheFreeSize=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:averageRegionSize=32082 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_95th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaMissCount=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_75th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:filteredReadRequestCount=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaHitCount=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_75th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_num_ops=1 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_95th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_median=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCount=18 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_num_ops=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.serverName=8c5d72df62de,35419,1697385888915 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReads=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowAppendCount=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:numReferenceFiles=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedFromMob=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSize=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_max=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_90th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_median=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_mean=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_98th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadWins=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseWarnThresholdExceeded=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_max=0 2023-10-15T16:14:16,606 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_25th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoHitCount=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_median=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreHeapSize=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_25th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_min=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99.9th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_25th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_num_ops=1 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_90th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_max=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99.9th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_max=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_median=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsSize=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_median=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_median=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_min=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkMissCount=1 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_95th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFreeSize=922061632 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheHitPercent=0.0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_98th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_num_ops=2 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.walReaderEditsBufferUsage=80117 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_25th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfHFileRefsQueue=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logReadInBytes=83847 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_75th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfLogQueue=-3 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_90th_percentile=0 2023-10-15T16:14:16,607 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99.9th_percentile=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_num_ops=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedOps=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_min=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedHFiles=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_min=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_mean=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBytes=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_90th_percentile=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedOps=1000 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedBatches=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_98th_percentile=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_max=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_95th_percentile=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedBatches=1 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_mean=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_95th_percentile=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedHFiles=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_median=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.uncleanlyClosedLogs=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99th_percentile=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.closedLogsWithUnknownFileLength=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_max=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.repeatedLogFileBytes=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedRecoverQueues=5 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Context=regionserver 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedRecoverQueues=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.restartedLogReading=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.ignoredUncleanlyClosedLogContentsInBytes=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsRead=40 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.numInitializing=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99.9th_percentile=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_75th_percentile=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsFiltered=38 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_25th_percentile=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedLogs=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.failedBatches=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBatches=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_median=0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,608 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWrite=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosNumOps=26 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRemoved=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsNumOps=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksCached=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsAvgTime=94.0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalNumOps=9 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesRead=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpAvgTime=6326.928571428572 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesAvgTime=0.0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionTasks=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncCount=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:NumProcessedCommands=10 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpNumOps=27 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromLocalClient=28 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:VolumeFailures=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosNumOps=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesRead=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosAvgTime=561936.0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalReadTime=9 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesRead=52810 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsAvgTime=1.7142857142857142 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromRemoteClient=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksUncached=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DatanodeNetworkErrors=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromLocalClient=27 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksInPendingIBR=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockVerificationFailures=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosNumOps=54 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesWrite=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosNumOps=54 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.SessionId=null 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsNumOps=1 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,609 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesWritten=620489 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRead=27 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReplicated=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Context=dfs 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksVerified=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromRemoteClient=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosAvgTime=20303.65 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsNumOps=9 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksWritten=28 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsAvgTime=1.0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalWriteTime=12 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcDecodingTimeNanos=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesNumOps=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosNumOps=1285 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpNumOps=28 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosAvgTime=23635.714285714286 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesWritten=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosAvgTime=10588.8 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpNumOps=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumFailedVolumes=0 2023-10-15T16:14:16,610 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheCapacity=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheUsed=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:FailedStorageLocations=[Ljava.lang.String;@4d7933d1 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksCached=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:LastVolumeFailureDate=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Capacity=1054589059072 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToUncache=7 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:DfsUsed=625022 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Remaining=941219363830 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToCache=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numRegionSizeReports=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_min=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_num_ops=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_98th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_num_ops=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numNamespaceInQuotaViolation=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_75th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_95th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_90th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_median=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_max=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.namespaceSpaceQuotaOverview=[] 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numTablesInQuotaViolation=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_num_ops=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99.9th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_max=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_98th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_median=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_min=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_max=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Context=master 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99.9th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_95th_percentile=0 2023-10-15T16:14:16,611 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_median=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_mean=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_mean=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_max=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_25th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_mean=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_min=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_75th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_mean=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_90th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_min=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_median=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.tableSpaceQuotaOverview=[] 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numSpaceQuotas=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_25th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_num_ops=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenSubmittedCount=14 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignSubmittedCount=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_num_ops=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99.9th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_median=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_max=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_95th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_95th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_min=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_mean=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_min=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_median=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_min=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveSubmittedCount=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_25th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_mean=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99.9th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_max=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnFS=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerRegions=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_75th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_95th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignSubmittedCount=13 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_75th_percentile=0 2023-10-15T16:14:16,612 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_98th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_num_ops=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_90th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignFailedCount=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_num_ops=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_90th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignFailedCount=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_25th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_98th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_98th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveFailedCount=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_mean=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_75th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeFailedCount=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Context=master 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_num_ops=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseSubmittedCount=1 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:holes=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_75th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99.9th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_95th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_95th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeSubmittedCount=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99.9th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_median=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99.9th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_mean=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCount=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenSubmittedCount=1 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseFailedCount=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_max=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:inconsistentRegions=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_90th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenFailedCount=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnRS=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99.9th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_75th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_75th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_min=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_25th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerOpenRegions=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_95th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_25th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:overlaps=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenFailedCount=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_90th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_98th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitFailedCount=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_max=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_98th_percentile=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_mean=0 2023-10-15T16:14:16,613 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_num_ops=13 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_median=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_mean=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCountOverThreshold=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_min=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_75th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_90th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_num_ops=1 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_75th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_95th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_25th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_median=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_25th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitSubmittedCount=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_mean=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritOldestAge=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:emptyRegionInfoRegions=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_max=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_25th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99.9th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_90th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_min=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99.9th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_num_ops=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_max=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_95th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_median=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_max=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_90th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_25th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_25th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_min=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_90th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_95th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_max=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_num_ops=1 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_min=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:operationCount=15 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_98th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:deadServerOpenRegions=3 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_mean=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_median=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_min=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99.9th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_98th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_median=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_max=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_mean=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_num_ops=14 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_98th_percentile=0 2023-10-15T16:14:16,614 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_90th_percentile=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_median=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_98th_percentile=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_75th_percentile=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumFailedVolumes=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheUsed=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheCapacity=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksCached=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:LastVolumeFailureDate=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Capacity=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Context=FSDatasetState 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:DfsUsed=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Remaining=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToCache=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToUnCache=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalDataFileIos=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalFileIoErrors=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateNumOps=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateNumOps=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateNumOps=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateNumOps=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateNumOps=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateNumOps=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Context=dfs 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalMetadataOperations=0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,615 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsNumOps=17 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportNumOps=14 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsNumOps=54 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoAvgTime=0.6666666666666666 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoNumOps=675 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestNumOps=3 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedNumOps=159 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateNumOps=60 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.port=39027 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeNumOps=3 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportAvgTime=0.3333333333333333 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Context=rpcdetailed 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameNumOps=41 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportAvgTime=0.21428571428571427 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateAvgTime=5.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteNumOps=60 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingNumOps=206 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsNumOps=322 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportNumOps=3 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeAvgTime=0.6666666666666666 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionNumOps=1 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockNumOps=49 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteNumOps=33 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateNumOps=1 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesNumOps=16 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeNumOps=1 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatNumOps=600 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseNumOps=19 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseNumOps=5 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseAvgTime=0.0 2023-10-15T16:14:16,616 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatAvgTime=0.0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemMaxM=2200.0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTerminated=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsWaiting=86 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.SessionId=null 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Context=jvm 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapUsedM=188.86484 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTotalExtraSleepTime=55 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Old Generation=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCount=11 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsBlocked=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogWarn=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsRunnable=83 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogInfo=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsNew=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapUsedM=327.47656 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTimedWaiting=119 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapMaxM=2200.0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Young Generation=11 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogError=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.ProcessName=DataNode 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogFatal=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillis=328 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDataNodeMessageCount=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastWrittenTransactionId=431 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:SnapshottableDirectories=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEncryptionZones=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReconstructionBlocks=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptBlocks=0 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastCheckpointTime=1697385856136 2023-10-15T16:14:16,617 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ReadLockLongHoldCount=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumActiveClients=1 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastLogRoll=431 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumTimedOutPendingReconstructions=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:VolumeFailuresTotal=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ProvidedCapacityTotal=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingECBlockGroups=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedGB=0.0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecommissioningDataNodes=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.HAState=active 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyECBlockGroups=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:FilesTotal=109 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ScheduledReplicationBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureECBlockGroups=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptECBlockGroups=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MillisSinceLastLoadedEdits=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalECBlockGroups=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExcessBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalLoad=9 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExpiredHeartbeats=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInServiceLiveDataNodes=3 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumStaleStorages=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:WriteLockLongHoldCount=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlocksTotal=42 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:UnderReplicatedBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicatedBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsed=1391792 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalReplicatedBlocks=42 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:StaleDataNodes=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumFilesUnderConstruction=1 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemaining=2823754896354 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PostponedMisreplicatedBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedNonDFS=178447369040 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LockQueueLength=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptReplicatedBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Context=dfs 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomDeadDataNodes=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:Snapshots=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionECBlocks=0 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.TotalSyncTimes=11 5 2023-10-15T16:14:16,618 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReplicationBlocks=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotal=3163767177216 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDeadDataNodes=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplOneBlocks=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalSyncCount=388 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastCheckpoint=431 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicationOneBlocks=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumLiveDataNodes=3 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlockCapacity=4194304 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotalGB=2946.0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyBlocks=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomLiveDataNodes=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemainingGB=2630.0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:ReceivedBytes=15456 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcSlowCalls=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationSuccesses=46 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumDroppedConnections=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcClientBackoff=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationFailures=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumOpenConnections=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.port=35085 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:SentBytes=9822 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeNumOps=46 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationSuccesses=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:CallQueueLength=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Context=rpc 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationFailures=0 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeNumOps=46 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeNumOps=46 2023-10-15T16:14:16,619 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:receivedBytes=288613 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationSuccesses=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_max=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFailures=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numOpenConnections=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_num_ops=1195 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_median=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationSuccesses=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_num_ops=1195 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_num_ops=1195 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_max=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_max=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_median=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.otherExceptions=5 2023-10-15T16:14:16,620 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Context=regionserver 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_median=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_min=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_median=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_max=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_min=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_min=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveHandler=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_num_ops=1195 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_mean=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReadHandler=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_median=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_num_ops=1195 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_mean=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callDropped=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions=5 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFallbacks=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:14:16,621 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationFailures=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_min=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_max=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:queueSize=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveScanHandler=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_min=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:sentBytes=44933 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_mean=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_75th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_max=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_90th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_min=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_median=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_num_ops=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOnHeapSize=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_num_ops=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_num_ops=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_median=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_num_ops=0 2023-10-15T16:14:16,622 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_75th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_min=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOffHeapSize=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_25th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_75th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_mean=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_25th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_25th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_mean=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tunerDoNothingCounter=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_25th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_num_ops=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:aboveHeapOccupancyLowWaterMarkCounter=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_98th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreSize=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_min=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_max=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_90th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_mean=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_75th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Context=regionserver 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99.9th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_mean=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_min=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockCacheSize=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_90th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockedFlushGauge=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_max=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_median=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_98th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:unblockedFlushGauge=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_max=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_max=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_num_ops=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_min=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_95th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_90th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_95th_percentile=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_max=0 2023-10-15T16:14:16,623 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_median=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_98th_percentile=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_min=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99.9th_percentile=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_median=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_median=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99th_percentile=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99th_percentile=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_95th_percentile=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_mean=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_95th_percentile=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_98th_percentile=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalDataFileIos=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalFileIoErrors=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateNumOps=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateNumOps=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateNumOps=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateNumOps=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateNumOps=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateNumOps=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Context=dfs 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalMetadataOperations=0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,624 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:HttpPort=null 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XceiverCount=3 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"2","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-15... 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:ClusterId=testClusterID 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test... 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DatanodeHostname=127.0.0.1 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SecurityEnabled=false 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:RpcPort=0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DataPort=0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SoftwareVersion=3.2.4 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SlowDisks=null 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XmitsInProgress=0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DNStartedTimeInMillis=1697385857021 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsBatchedInSync=44 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsNumOps=387 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsAvgTime=0.11764705882352941 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportNumOps=0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeNumOps=0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteSnapshotOps=0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsAvgTime=0.09523809523809523 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AllowSnapshotOps=0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:RenameSnapshotOps=0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeAvgTime=0.08333333333333333 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesCreated=168 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesAppended=0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportAvgTime=0.0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageAvgTime=0.0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeNumOps=0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeNumOps=0 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeNumOps=126 2023-10-15T16:14:16,626 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeAvgTime=0.0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumTimesReReplicationNotScheduled=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SafeModeTime=1 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalNumOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetListingOps=132 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.ProcessName=NameNode 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageNumOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TotalFileOps=1303 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSymlinkOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AddBlockOps=49 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DisallowSnapshotOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteFileOps=23 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsNumOps=431 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSnapshotOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PendingEditsCount=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ListSnapshottableDirOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockReceivedAndDeletedOps=159 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.SessionId=null 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditNumOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FsImageLoadTime=23 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateFileOps=60 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Context=dfs 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesRenamed=41 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TimeoutReReplications=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsQueued=1 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedNumOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageNumOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageAvgTime=0.0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesInGetListingOps=146 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FileInfoOps=675 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetLinkTargetOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetAdditionalDatanodeOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsBatched=68 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesDeleted=39 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SuccessfulReReplications=21 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetBlockLocations=322 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesTruncated=1 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SnapshotDiffReportOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportNumOps=6 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeNumOps=0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditAvgTime=0.0 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportAvgTime=0.3333333333333333 2023-10-15T16:14:16,627 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumFailedVolumes=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheUsed=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheCapacity=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksCached=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:LastVolumeFailureDate=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Capacity=1054589059072 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Context=FSDatasetState 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:DfsUsed=383910 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Remaining=941226220534 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToCache=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToUnCache=7 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalDataFileIos=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalFileIoErrors=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateNumOps=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateNumOps=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateNumOps=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateNumOps=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateNumOps=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateNumOps=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Context=dfs 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalMetadataOperations=0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessNumOps=1 2023-10-15T16:14:16,628 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureNumOps=0 2023-10-15T16:14:16,629 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailures=0 2023-10-15T16:14:16,629 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsNumOps=2 2023-10-15T16:14:16,629 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailuresTotal=0 2023-10-15T16:14:16,629 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Context=ugi 2023-10-15T16:14:16,629 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsAvgTime=6.0 2023-10-15T16:14:16,629 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,629 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessAvgTime=10.0 2023-10-15T16:14:16,629 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureAvgTime=0.0 2023-10-15T16:14:16,640 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfSnapshottableDirs=0 2023-10-15T16:14:16,640 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheCapacity=0 2023-10-15T16:14:16,640 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheUsed=0 2023-10-15T16:14:16,640 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ProvidedCapacity=0 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DistinctVersionCount=1 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocksWithReplicationFactorOne=0 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentRemaining=89.25292 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentUsed=4.3991608E-5 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:TotalBlocks=42 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NonDfsUsedSpace=178447369040 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Safemode= 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:UpgradeFinalized=true 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:EnteringMaintenanceNodes={} 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentBlockPoolUsed=4.3991608E-5 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:LiveNodes={"127.0.0.1:40557":{"infoAddr":"127.0.0.1:35357","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:40557","lastContact":2,"us... 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CompileInfo=2022-07-12T11:58Z by ubuntu from branch-3.2.4 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolId=BP-1531809096-172.17.0.3-1697385856100 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ClusterId=testClusterID 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DeadNodes={} 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:JournalTransactionInfo={"MostRecentCheckpointTxId":"0","LastAppliedOrWrittenTxId":"431"} 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Free=2823754896354 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolUsedSpace=1391792 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Threads=288 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocks=0 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirStatuses={"active":{"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/t... 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:SoftwareVersion=3.2.4 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:VerifyECWithTopologyResult={"isSupported":"false","resultMessage":"The number of DataNodes (3) is less than the minimum required number of DataNodes (9) fo... 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Used=1391792 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Total=3163767177216 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NodeUsage={"nodeUsage":{"min":"0.00%","median":"0.00%","max":"0.00%","stdDev":"0.00%"}} 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NNStartedTimeInMillis=1697385856352 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DecomNodes={} 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CorruptFiles=[] 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirSize={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test... 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameJournalStatus=[{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3... 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemMaxM=2200.0 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTerminated=0 2023-10-15T16:14:16,641 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsWaiting=86 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.SessionId=null 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Context=jvm 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapUsedM=188.87222 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTotalExtraSleepTime=36 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Old Generation=0 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCount=11 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsBlocked=0 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogWarn=0 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsRunnable=83 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogInfo=0 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsNew=0 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapUsedM=328.47656 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTimedWaiting=119 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapMaxM=2200.0 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Young Generation=11 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogError=0 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.ProcessName=DataNode 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogFatal=0 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillis=328 2023-10-15T16:14:16,642 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,666 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlocksTotal=42 2023-10-15T16:14:16,666 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TopUserOpCounts={"timestamp":"2023-10-15T16:14:16+0000","windows":[{"ops":[{"opType":"*","totalCount":1,"allUsers":[{"user":"jenkins","count":1}... 2023-10-15T16:14:16,666 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:SnapshotStats={"SnapshottableDirectories":0,"Snapshots":0} 2023-10-15T16:14:16,666 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:UnderReplicatedBlocks=0 2023-10-15T16:14:16,666 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:16,666 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityUsed=1391792 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEncryptionZones=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityRemaining=2823754896354 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReconstructionBlocks=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomDeadDataNodes=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FSState=Operational 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:VolumeFailuresTotal=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ProvidedCapacityTotal=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecommissioningDataNodes=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReplicationBlocks=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncTimes=11 5 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingDeletionBlocks=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityTotal=3163767177216 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FilesTotal=109 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ScheduledReplicationBlocks=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDeadDataNodes=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncCount=388 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FsLockQueueLength=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:MaxObjects=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalLoad=9 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleDataNodes=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumLiveDataNodes=3 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlockDeletionStartTime=1697385856352 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInServiceLiveDataNodes=3 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:LowRedundancyBlocks=0 2023-10-15T16:14:16,667 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(83): >> Begin DataNode metrics dump 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomLiveDataNodes=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleStorages=0 2023-10-15T16:14:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(109): << End NameNode metrics dump 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_num_ops=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_mean=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_num_ops=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_25th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_95th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_90th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_90th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_max=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_98th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_25th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_75th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_min=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_75th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_min=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_max=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_max=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_25th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99.9th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_75th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_mean=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99.9th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_75th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_median=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_95th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_98th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99.9th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_median=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_25th_percentile=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Context=master 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_median=0 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,680 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_95th_percentile=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_98th_percentile=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_90th_percentile=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_mean=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99.9th_percentile=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_95th_percentile=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_min=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_max=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99th_percentile=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_98th_percentile=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_mean=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_num_ops=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_min=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_median=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_num_ops=0 2023-10-15T16:14:16,681 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_90th_percentile=0 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:HttpPort=null 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XceiverCount=3 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"2","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"599"}] 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:ClusterId=testClusterID 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1":{"numBlocks":21,"usedSpace":266240,"freeSpace":470676213760,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2":{"numBlocks":21,"usedSpace":117670,"freeSpace":470542240758,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":133973002}} 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DatanodeHostname=127.0.0.1 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SecurityEnabled=false 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:RpcPort=0 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DataPort=0 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SoftwareVersion=3.2.4 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SlowDisks=null 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XmitsInProgress=0 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DNStartedTimeInMillis=1697385856691 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Context=regionserver 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.port=44717 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Context=rpcdetailed 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthNumOps=50 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpNumOps=4 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWrite=0 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,682 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosNumOps=44 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRemoved=7 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalAvgTime=0.9795918367346939 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsNumOps=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksCached=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsAvgTime=2.0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DataNodeActiveXceiversCount=1 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalNumOps=200 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsNumOps=53 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesRead=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpAvgTime=3792.75 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesAvgTime=0.0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionTasks=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncCount=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:NumProcessedCommands=201 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpNumOps=92 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromLocalClient=48 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:VolumeFailures=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosNumOps=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesRead=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosAvgTime=1433727.0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalReadTime=16 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesRead=298317 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsAvgTime=1.0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromRemoteClient=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksUncached=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DatanodeNetworkErrors=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromLocalClient=92 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksInPendingIBR=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockVerificationFailures=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosNumOps=184 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesWrite=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosNumOps=184 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.SessionId=null 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsNumOps=1 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesWritten=973747 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,683 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRead=92 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReplicated=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Context=dfs 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksVerified=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromRemoteClient=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosAvgTime=20216.833333333332 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsNumOps=200 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksWritten=48 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsAvgTime=0.9183673469387755 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalWriteTime=21 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcDecodingTimeNanos=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesNumOps=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosNumOps=1700 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpNumOps=48 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosAvgTime=5276.454545454545 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesWritten=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosAvgTime=12045.5 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpNumOps=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumFailedVolumes=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheUsed=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheCapacity=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksCached=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:LastVolumeFailureDate=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Capacity=1054589059072 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Context=FSDatasetState 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:DfsUsed=625022 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Remaining=941225311222 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToCache=0 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToUnCache=7 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheCleared=1 2023-10-15T16:14:16,684 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheHit=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Context=rpc 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheUpdated=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalDataFileIos=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalFileIoErrors=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateNumOps=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateNumOps=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateNumOps=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateNumOps=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateNumOps=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateNumOps=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Context=dfs 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalMetadataOperations=0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,685 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:MissingECBlockGroups=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:LowRedundancyECBlockGroups=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:BytesInFutureECBlockGroups=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:CorruptECBlockGroups=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:PendingDeletionECBlocks=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:TotalECBlockGroups=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheCleared=1 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheHit=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Context=rpc 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheUpdated=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:LastHATransitionTime=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:HostAndPort=localhost:39027 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:State=active 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowPeersReport=null 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowDisksReport=null 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:NNRole=NameNode 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:BytesWithFutureGenerationStamps=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SecurityEnabled=false 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.port=35753 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Context=rpcdetailed 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthNumOps=55 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:ReceivedBytes=18480 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcSlowCalls=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationSuccesses=55 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumDroppedConnections=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcClientBackoff=0 2023-10-15T16:14:16,686 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationFailures=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumOpenConnections=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.port=35753 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:SentBytes=11751 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeNumOps=55 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationSuccesses=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:CallQueueLength=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Context=rpc 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationFailures=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeNumOps=55 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeNumOps=55 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:ReceivedBytes=16800 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcSlowCalls=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationSuccesses=50 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumDroppedConnections=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcClientBackoff=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationFailures=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumOpenConnections=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.port=44717 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:SentBytes=10673 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeNumOps=50 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationSuccesses=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:CallQueueLength=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Context=rpc 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationFailures=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeNumOps=50 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeNumOps=50 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemMaxM=2200.0 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,687 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTerminated=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsWaiting=86 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.SessionId=null 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Context=jvm 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapUsedM=188.82245 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTotalExtraSleepTime=51 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Old Generation=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCount=11 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsBlocked=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogWarn=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsRunnable=83 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogInfo=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsNew=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapUsedM=323.47656 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTimedWaiting=119 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapMaxM=2200.0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Young Generation=11 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogError=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.ProcessName=NameNode 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogFatal=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillis=328 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumFailedVolumes=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheUsed=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheCapacity=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksCached=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:LastVolumeFailureDate=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Capacity=1054589059072 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Context=FSDatasetState 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:DfsUsed=382860 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Remaining=941225311222 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToCache=0 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToUnCache=7 2023-10-15T16:14:16,688 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemMaxM=2200.0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTerminated=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsWaiting=86 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.SessionId=null 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Context=jvm 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapUsedM=188.82245 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTotalExtraSleepTime=22 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Old Generation=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCount=11 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsBlocked=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogWarn=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsRunnable=84 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogInfo=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsNew=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapUsedM=323.47656 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTimedWaiting=118 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapMaxM=2200.0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Young Generation=11 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogError=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.ProcessName=DataNode 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogFatal=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillis=328 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_num_ops=1891 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99th_percentile=1 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_max=4 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_mean=206 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:appendCount=1891 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:lowReplicaRollRequest=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99th_percentile=332 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99.9th_percentile=4 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:sizeRollRequest=0 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendSize=297021 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_max=1 2023-10-15T16:14:16,689 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowAppendCount=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:successfulLogRolls=24 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99.9th_percentile=333 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_TimeRangeCount_0-1=11 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_25th_percentile=36 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendSize=9557 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_90th_percentile=1 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendCount=1011 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_25th_percentile=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_10-100=4 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99th_percentile=4 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_98th_percentile=4 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_75th_percentile=298 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_75th_percentile=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_max=334 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_95th_percentile=4 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_mean=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendCount=48 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_100-1000=7 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:errorRollRequest=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:rollRequest=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_num_ops=1804 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_median=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_3-10=1 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_98th_percentile=1 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_min=36 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_min=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_median=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Context=regionserver 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_0-1=10 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendCount=45 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_95th_percentile=326 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendSize=74104 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_median=262 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_98th_percentile=331 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_25th_percentile=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_75th_percentile=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_mean=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99.9th_percentile=1 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendSize=14494 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:writtenBytes=395176 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_90th_percentile=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_95th_percentile=1 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowSyncRollRequest=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_num_ops=1891 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_90th_percentile=319 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_min=0 2023-10-15T16:14:16,690 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendCount=787 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicationOneBlocks=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:CorruptReplicatedBlocks=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicatedBlocks=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:TotalReplicatedBlocks=42 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalDataFileIos=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalFileIoErrors=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateNumOps=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateNumOps=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateNumOps=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateNumOps=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateNumOps=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateNumOps=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Context=dfs 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalMetadataOperations=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotNumOps=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishNumOps=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotAvgTime=0.0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Context=metricssystem 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSources=67 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:DroppedPubAll=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishAvgTime=0.0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSources=67 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSinks=0 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,691 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSinks=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheCleared=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheHit=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Context=rpc 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheUpdated=134 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWrite=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosNumOps=28 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRemoved=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsNumOps=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksCached=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsAvgTime=3.0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalNumOps=9 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesRead=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpAvgTime=9.0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpAvgTime=6810.7692307692305 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesAvgTime=0.0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionTasks=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncCount=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:NumProcessedCommands=10 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpNumOps=31 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromLocalClient=28 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:VolumeFailures=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosNumOps=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesRead=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,692 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosAvgTime=554484.75 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalReadTime=7 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesRead=57958 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsAvgTime=1.5714285714285714 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromRemoteClient=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksUncached=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DatanodeNetworkErrors=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromLocalClient=31 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksInPendingIBR=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockVerificationFailures=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosNumOps=62 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesWrite=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosNumOps=62 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.SessionId=null 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsNumOps=1 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesWritten=620489 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRead=31 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReplicated=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Context=dfs 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksVerified=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromRemoteClient=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosAvgTime=19658.18181818182 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsNumOps=9 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksWritten=28 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsAvgTime=1.0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalWriteTime=12 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcDecodingTimeNanos=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesNumOps=0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosNumOps=1285 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpNumOps=28 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosAvgTime=4301.205882352941 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,693 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesWritten=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosAvgTime=9971.454545454546 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpAvgTime=0.8181818181818182 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpNumOps=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemMaxM=2200.0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTerminated=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsWaiting=86 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.SessionId=null 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Context=jvm 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapUsedM=188.83234 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTotalExtraSleepTime=62 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Old Generation=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCount=11 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsBlocked=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogWarn=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsRunnable=83 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogInfo=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsNew=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapUsedM=323.47656 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTimedWaiting=119 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapMaxM=2200.0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Young Generation=11 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogError=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.ProcessName=DataNode 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogFatal=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillis=328 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Context=regionserver 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,694 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:numTables=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99.9th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_90th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_95th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterRequests=1101 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterWriteRequests=1059 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_mean=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterReadRequests=74 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_min=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_25th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashSubmittedCount=4 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashFailedCount=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_98th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Context=master 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_num_ops=2 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_75th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_max=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_median=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:receivedBytes=6048147 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationSuccesses=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_max=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFailures=0 2023-10-15T16:14:16,695 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numOpenConnections=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_num_ops=1266 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_median=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationSuccesses=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_num_ops=1266 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_num_ops=1266 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_max=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_max=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_median=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.otherExceptions=3 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Context=master 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_median=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_min=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_median=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_max=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_min=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_min=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveHandler=0 2023-10-15T16:14:16,696 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_num_ops=1266 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_mean=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReadHandler=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_median=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_num_ops=1266 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_mean=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callDropped=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions=3 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFallbacks=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationFailures=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_min=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_max=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:queueSize=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveScanHandler=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_min=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:sentBytes=16055 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,697 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumFailedVolumes=0 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheUsed=0 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheCapacity=0 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksCached=0 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:LastVolumeFailureDate=0 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Capacity=0 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Context=FSDatasetState 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data4]'} 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:DfsUsed=0 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Remaining=0 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToCache=0 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToUnCache=0 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:HttpPort=null 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XceiverCount=3 2023-10-15T16:14:16,698 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"1","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"598"}] 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:ClusterId=testClusterID 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5":{"numBlocks":21,"usedSpace":266240,"freeSpace":470675873792,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6":{"numBlocks":21,"usedSpace":116620,"freeSpace":470541900790,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":133973002}} 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DatanodeHostname=127.0.0.1 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SecurityEnabled=false 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:RpcPort=0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DataPort=0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SoftwareVersion=3.2.4 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SlowDisks=null 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XmitsInProgress=0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DNStartedTimeInMillis=1697385857417 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalDataFileIos=0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalFileIoErrors=0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateNumOps=0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,699 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateNumOps=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateNumOps=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateNumOps=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateNumOps=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateNumOps=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Context=dfs 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalMetadataOperations=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumFailedVolumes=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheUsed=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheCapacity=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksCached=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:LastVolumeFailureDate=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Capacity=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Context=FSDatasetState 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data6]'} 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:DfsUsed=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Remaining=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToCache=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToUnCache=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:ReceivedBytes=744936 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeAvgTime=0.009771986970684038 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcSlowCalls=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationSuccesses=27 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumDroppedConnections=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.NumOpenConnectionsPerUser={"jenkins":1} 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcClientBackoff=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeAvgTime=0.006514657980456026 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationFailures=0 2023-10-15T16:14:16,700 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumOpenConnections=1 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.port=39027 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:SentBytes=627674 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeNumOps=2342 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationSuccesses=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:CallQueueLength=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Context=rpc 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationFailures=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeNumOps=2342 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.serverName=ClientNamenodeProtocol 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeNumOps=2342 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.port=35085 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Context=rpcdetailed 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthNumOps=46 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemMaxM=2200.0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTerminated=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsWaiting=86 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.SessionId=null 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Context=jvm 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapUsedM=188.845 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTotalExtraSleepTime=30 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Old Generation=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCount=11 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsBlocked=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogWarn=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsRunnable=83 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogInfo=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsNew=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapUsedM=323.47656 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTimedWaiting=119 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapMaxM=2200.0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Young Generation=11 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogError=0 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.ProcessName=DataNode 2023-10-15T16:14:16,701 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogFatal=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillis=328 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsBatchedInSync=31 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsNumOps=224 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsAvgTime=0.13043478260869565 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportNumOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeNumOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteSnapshotOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsAvgTime=0.03896103896103896 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AllowSnapshotOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:RenameSnapshotOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeAvgTime=0.0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesCreated=80 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesAppended=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportAvgTime=0.0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageAvgTime=0.0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeNumOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeNumOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeNumOps=3 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeAvgTime=0.0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumTimesReReplicationNotScheduled=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SafeModeTime=3 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalNumOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetListingOps=66 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.ProcessName=NameNode 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageNumOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TotalFileOps=540 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSymlinkOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AddBlockOps=28 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DisallowSnapshotOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteFileOps=5 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsNumOps=255 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSnapshotOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PendingEditsCount=1 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ListSnapshottableDirOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockReceivedAndDeletedOps=96 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.SessionId=null 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditNumOps=0 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FsImageLoadTime=173 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateFileOps=36 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Context=dfs 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesRenamed=17 2023-10-15T16:14:16,702 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TimeoutReReplications=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsQueued=1 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedNumOps=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageNumOps=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageAvgTime=0.0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesInGetListingOps=35 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FileInfoOps=270 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetLinkTargetOps=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetAdditionalDatanodeOps=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsBatched=9 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesDeleted=1 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SuccessfulReReplications=2 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetBlockLocations=118 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesTruncated=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SnapshotDiffReportOps=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportNumOps=6 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeNumOps=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditAvgTime=0.0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportAvgTime=0.5 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumFailedVolumes=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheCapacity=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheUsed=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:FailedStorageLocations=[Ljava.lang.String;@21dbbc70 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksCached=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:LastVolumeFailureDate=0 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Capacity=1054589059072 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:14:16,703 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToUncache=7 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:DfsUsed=383910 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Remaining=941217381366 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToCache=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpNumOps=5 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWrite=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosNumOps=48 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRemoved=7 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalAvgTime=1.0303030303030303 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsNumOps=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksCached=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsAvgTime=2.0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DataNodeActiveXceiversCount=1 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalNumOps=200 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsNumOps=51 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesRead=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpAvgTime=3792.5 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesAvgTime=0.0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionTasks=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncCount=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:NumProcessedCommands=201 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpNumOps=99 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromLocalClient=48 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:VolumeFailures=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosNumOps=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesRead=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosAvgTime=388889.5 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalReadTime=29 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesRead=688752 2023-10-15T16:14:16,704 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsAvgTime=0.75 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromRemoteClient=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksUncached=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DatanodeNetworkErrors=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromLocalClient=99 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksInPendingIBR=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockVerificationFailures=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosNumOps=200 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesWrite=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosNumOps=200 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.SessionId=null 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsNumOps=1 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesWritten=973747 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRead=99 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReplicated=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Context=dfs 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksVerified=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromRemoteClient=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosAvgTime=45642.0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsNumOps=200 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksWritten=48 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsAvgTime=0.9797979797979798 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalWriteTime=22 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcDecodingTimeNanos=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesNumOps=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosNumOps=1700 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpNumOps=48 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosAvgTime=4914.0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesWritten=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosAvgTime=9294.0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpNumOps=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,705 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWrite=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosNumOps=30 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRemoved=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsNumOps=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksCached=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsAvgTime=95.0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalNumOps=9 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesRead=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpAvgTime=6809.0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesAvgTime=0.0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionTasks=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncCount=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:NumProcessedCommands=10 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpNumOps=35 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromLocalClient=28 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:VolumeFailures=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosNumOps=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesRead=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosAvgTime=352052.375 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalReadTime=16 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesRead=75811 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsAvgTime=1.3846153846153846 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromRemoteClient=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksUncached=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DatanodeNetworkErrors=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromLocalClient=35 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksInPendingIBR=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockVerificationFailures=0 2023-10-15T16:14:16,706 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosNumOps=70 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesWrite=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosNumOps=70 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.SessionId=null 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsNumOps=1 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesWritten=620489 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRead=35 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReplicated=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Context=dfs 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksVerified=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromRemoteClient=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosAvgTime=25843.916666666668 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsNumOps=9 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksWritten=28 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsAvgTime=1.0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalWriteTime=12 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcDecodingTimeNanos=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesNumOps=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosNumOps=1285 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpNumOps=28 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosAvgTime=4740.882352941177 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesWritten=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosAvgTime=9895.041666666666 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpAvgTime=0.75 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpNumOps=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,707 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpNumOps=4 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWrite=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosNumOps=32 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRemoved=7 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalAvgTime=0.9795918367346939 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsNumOps=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksCached=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsAvgTime=2.0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DataNodeActiveXceiversCount=1 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalNumOps=200 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsNumOps=55 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesRead=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpAvgTime=3793.25 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesAvgTime=0.0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionTasks=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncCount=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:NumProcessedCommands=201 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpNumOps=89 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromLocalClient=48 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:VolumeFailures=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosNumOps=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesRead=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosAvgTime=304427.5 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalReadTime=12 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesRead=339498 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsAvgTime=0.75 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromRemoteClient=0 2023-10-15T16:14:16,708 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksUncached=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DatanodeNetworkErrors=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromLocalClient=89 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksInPendingIBR=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockVerificationFailures=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosNumOps=178 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesWrite=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosNumOps=178 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.SessionId=null 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsNumOps=1 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesWritten=973747 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRead=89 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReplicated=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Context=dfs 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksVerified=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromRemoteClient=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosAvgTime=17297.333333333332 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsNumOps=200 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksWritten=48 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsAvgTime=0.9489795918367347 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalWriteTime=28 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcDecodingTimeNanos=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesNumOps=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosNumOps=1700 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpNumOps=48 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosAvgTime=4986.545454545455 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesWritten=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosAvgTime=11669.166666666666 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpAvgTime=0.3333333333333333 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpNumOps=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,709 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDataNodeMessageCount=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastWrittenTransactionId=255 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:SnapshottableDirectories=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEncryptionZones=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReconstructionBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastCheckpointTime=1697385845476 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ReadLockLongHoldCount=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumActiveClients=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastLogRoll=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumTimedOutPendingReconstructions=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:VolumeFailuresTotal=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ProvidedCapacityTotal=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingECBlockGroups=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedGB=0.0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecommissioningDataNodes=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.HAState=active 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyECBlockGroups=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:FilesTotal=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ScheduledReplicationBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureECBlockGroups=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptECBlockGroups=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MillisSinceLastLoadedEdits=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalECBlockGroups=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExcessBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalLoad=32 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExpiredHeartbeats=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInServiceLiveDataNodes=3 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumStaleStorages=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:WriteLockLongHoldCount=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlocksTotal=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:UnderReplicatedBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicatedBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsed=495713 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalReplicatedBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:StaleDataNodes=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumFilesUnderConstruction=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemaining=2816043276636 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PostponedMisreplicatedBlocks=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedNonDFS=185085906847 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LockQueueLength=0 2023-10-15T16:14:16,710 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptReplicatedBlocks=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingBlocks=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Context=dfs 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomDeadDataNodes=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:Snapshots=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionECBlocks=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.TotalSyncTimes= 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReplicationBlocks=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotal=3163767177216 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDeadDataNodes=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplOneBlocks=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalSyncCount=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastCheckpoint=255 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicationOneBlocks=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumLiveDataNodes=3 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlockCapacity=4194304 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotalGB=2946.0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyBlocks=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomLiveDataNodes=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemainingGB=2623.0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Context=regionserver 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheCleared=1 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheHit=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Context=rpc 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheUpdated=65 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemMaxM=2200.0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTerminated=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsWaiting=86 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.SessionId= 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Context=jvm 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapUsedM=188.84634 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Old Generation=0 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCount=11 2023-10-15T16:14:16,711 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsBlocked=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogWarn=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsRunnable=83 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogInfo=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsNew=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapUsedM=324.47656 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTimedWaiting=119 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapMaxM=2200.0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Young Generation=11 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogError=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.ProcessName=Master 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogFatal=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillis=328 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsCount=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeTotal=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointPercentComplete=1.0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageTotal=1 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsTotal=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointCount=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:ElapsedTime=681 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsPercentComplete=1.0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsElapsedTime=7 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointElapsedTime=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModePercentComplete=1.0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:PercentComplete=1.0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointTotal=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeCount=0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeElapsedTime=13 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageElapsedTime=62 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageCount=1 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImagePercentComplete=1.0 2023-10-15T16:14:16,712 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumFailedVolumes=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheCapacity=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheUsed=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:FailedStorageLocations=[Ljava.lang.String;@2f1c563b 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksCached=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:LastVolumeFailureDate=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Capacity=1054589059072 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToUncache=7 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:DfsUsed=382860 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Remaining=941216979958 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToCache=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsCount=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeTotal=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointPercentComplete=1.0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageTotal=1 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsTotal=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointCount=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:ElapsedTime=681 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsPercentComplete=1.0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsElapsedTime=7 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointElapsedTime=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModePercentComplete=1.0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:PercentComplete=1.0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointTotal=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeCount=0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeElapsedTime=13 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageElapsedTime=62 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageCount=1 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImagePercentComplete=1.0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RackLocalityCostFunction=0.0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionCountSkewCostFunction=0.0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaHostCostFunction=0.0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ServerLocalityCostFunction=0.0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MemStoreSizeCostFunction=0.0 2023-10-15T16:14:16,713 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_95th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_Overall=0.0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ReadRequestCostFunction=0.0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_25th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_98th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_min=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Context=master 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_WriteRequestCostFunction=0.0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaRackCostFunction=0.0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_PrimaryRegionCountSkewCostFunction=0.0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:miscInvocationCount=18 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_median=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_num_ops=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_75th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MoveCostFunction=0.0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_max=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_mean=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_CPRequestCostFunction=0.0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_TableSkewCostFunction=0.0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_StoreFileCostFunction=0.0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_90th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.isBalancerActive=true 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99.9th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_95th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99.9th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_median=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_mean=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_25th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_max=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_95th_percentile=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:16,714 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_num_ops=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_25th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_98th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_min=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_mean=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_90th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_75th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_90th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_mean=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_num_ops=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_95th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_min=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_median=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_num_ops=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_mean=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_98th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_75th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_median=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99.9th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Context=regionserver 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_max=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_num_ops=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_num_ops=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_min=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_90th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_mean=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_95th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_max=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_75th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_75th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_min=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_num_ops=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99th_percentile=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:16,715 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_75th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_max=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_min=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_98th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_90th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_max=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_98th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_90th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_75th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_95th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_25th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_max=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_max=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_98th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_num_ops=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99.9th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_min=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_98th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_95th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_min=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_median=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:BlockBytesScannedCount=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99.9th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_25th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_median=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_90th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_95th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_num_ops=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_max=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_mean=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_max=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,716 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_min=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_mean=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_95th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99.9th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_25th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99.9th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_90th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_mean=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_95th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_min=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_75th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_max=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_max=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_90th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_max=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_98th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_25th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_median=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_median=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_75th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_mean=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99.9th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_median=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_90th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_mean=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_mean=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_75th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_num_ops=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99.9th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_median=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_75th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_median=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_median=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_98th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_95th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_75th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_95th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_num_ops=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_max=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_98th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_25th_percentile=0 2023-10-15T16:14:16,717 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_95th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99.9th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_num_ops=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_25th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99.9th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_median=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_90th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_25th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_median=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_min=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_max=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_90th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_max=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_75th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_25th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_98th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_min=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_mean=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99.9th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_25th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_min=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_mean=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_min=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_98th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_min=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_98th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99.9th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_median=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_num_ops=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_median=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_90th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_min=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_25th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,718 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99th_percentile=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsCount=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_98th_percentile=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_median=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestRatePerSecond=0.0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerMissCount=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_mean=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99th_percentile=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_median=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_98th_percentile=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_95th_percentile=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_median=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,719 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_90th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_5min_rate=0.0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_min=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_95th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitRatio=0.0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_75th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_90th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_num_ops=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_min=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_25th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestCount=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedOutputBytes=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_90th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkHitCount=1 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadOpsInCurThread=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_median=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestRatePerSecond=0.0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_25th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_min=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_num_ops=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_num_ops=26 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_90th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockBytesScannedCount=4389 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_max=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_75th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_max=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_mean=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_num_ops=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_25th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheEvictionCount=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99.9th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheCount=3 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:smallCompactionQueueLength=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCount=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_min=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_num_ops=15 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_mean=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowIncrementCount=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushQueueLength=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,720 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_mean=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_75th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:localBytesRead=108773 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushCount=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_median=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutateFailedCount=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_max=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorPoolAllocationBytes=1530880 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_98th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_min=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_median=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_min=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:updatesBlockedTime=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_median=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_mean=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitQueueLength=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_min=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_min=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_95th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCachingCount=3 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_25th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99.9th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileAge=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_median=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_75th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_max=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_95th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_num_ops=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_min=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cpRequestCount=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_90th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_median=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_max=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_num_ops=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALCount=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_98th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitRatio=0.7272727272727273 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataMissCount=1 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestCount=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissCount=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_75th_percentile=0 2023-10-15T16:14:16,721 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_mean=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheEvictionCount=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheFreeSize=922067336 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_95th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_median=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_75th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_max=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_median=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:activeScanners=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissCount=3 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_mean=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_max=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_max=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_95th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactionQueueLength=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_95th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_num_ops=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_max=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_1min_rate=0.0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_min=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_median=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_num_ops=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_median=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_mean=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_75th_percentile=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_min=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:16,722 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99.9th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_median=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_25th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_25th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_90th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_num_ops=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheSize=679544 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_num_ops=6 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_num_ops=26 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_98th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_count=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_median=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowGetCount=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_median=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterRequestsCount=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_min=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_min=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_25th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_max=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileSize=414534 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_min=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileCount=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsCount=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99.9th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_98th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_num_ops=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedOutputBytes=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_max=0 2023-10-15T16:14:16,723 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_num_ops=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_max=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticBloomSize=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexMissCount=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseInfoThresholdExceeded=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCount=3 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCount=3 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_max=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedToMob=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedMemstoreBytes=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheExpressHitPercent=72.72727272727273 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionServerStartTime=1697385859285 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedInputBytes=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedFromMob=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_max=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRequestCount=47 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_25th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcGetRequestCount=3 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_90th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeCount=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCountHitPercent=72.72727272727273 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_max=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_mean=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_min=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_98th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_min=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsSize=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_mean=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowDeleteCount=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockedRequestCount=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_mean=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheAccessCount=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_min=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_median=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_min=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_90th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_min=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_95th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_90th_percentile=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:16,724 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheEvictedCount=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_90th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_max=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:avgStoreFileAge=1819 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_25th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheCount=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitRequestCount=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_num_ops=1 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_min=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalBytesRead=108773 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedToMob=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_max=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedInputBytes=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheSize=679544 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileCount=473 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_mean=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcScanRequestCount=26 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreOffHeapSize=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_75th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheMissCount=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowPutCount=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_max=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreSize=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_75th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsSize=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_mean=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_min=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_75th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_median=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_98th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_median=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Context=regionserver 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:scannerLeaseExpiredCount=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:minStoreFileAge=1782 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitCount=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_98th_percentile=0 2023-10-15T16:14:16,725 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcFullScanRequestCount=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_mean=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitSuccessCount=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_mean=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_75th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_25th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCountPrimary=3 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_max=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataBlockCount=1 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_98th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_max=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_median=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsCount=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_mean=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsCount=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCachingCount=8 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFailedInsertionCount=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_90th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsSize=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean_rate=0.0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticIndexSize=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorUsedBufferCount=2 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_min=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99.9th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissRatio=0.2727272727272727 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_75th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_max=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_mean=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_95th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMultiRequestCount=1 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheSize=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexHitCount=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_max=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_98th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedOutputBytes=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocal=0.0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,726 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_98th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_max=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_num_ops=3 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99.9th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_25th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_min=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_mean=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_15min_rate=0.0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99.9th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_min=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_95th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_75th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_median=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_median=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_mean=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_95th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitCount=8 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:largeCompactionQueueLength=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_mean=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutatePassedCount=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_max=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_95th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissRatio=0.0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsCount=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_min=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_mean=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_min=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:shortCircuitBytesRead=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99.9th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:16,727 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_median=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_min=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoMissCount=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileIndexSize=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99.9th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_mean=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerHitCount=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_98th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsSize=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCountPrimary=8 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_min=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_max=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99.9th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_90th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_90th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_num_ops=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_max=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_mean=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_max=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_min=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRowActionRequestCount=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMutateRequestCount=16 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheCount=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_25th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_mean=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataHitCount=7 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:zeroCopyBytesRead=0 2023-10-15T16:14:16,728 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_98th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionCount=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_num_ops=1 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileCount=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_num_ops=1 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_median=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALSize=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_num_ops=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_mean=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_median=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheFreeSize=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:averageRegionSize=17474 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_95th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaMissCount=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_75th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:filteredReadRequestCount=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaHitCount=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_75th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_num_ops=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_95th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_median=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCount=8 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_num_ops=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.serverName=8c5d72df62de,45999,1697385859285 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReads=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowAppendCount=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:numReferenceFiles=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedFromMob=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSize=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_max=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_90th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_median=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_mean=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_98th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadWins=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseWarnThresholdExceeded=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_max=0 2023-10-15T16:14:16,729 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_25th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoHitCount=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_median=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreHeapSize=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_25th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_min=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99.9th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_25th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_num_ops=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_90th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_max=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99.9th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_max=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_median=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsSize=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_median=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_median=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_min=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkMissCount=2 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_95th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFreeSize=922067336 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheHitPercent=0.0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemMaxM=2200.0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTerminated=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsWaiting=86 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.SessionId=null 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Context=jvm 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapUsedM=188.85072 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTotalExtraSleepTime=30 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Old Generation=0 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCount=11 2023-10-15T16:14:16,730 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsBlocked=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogWarn=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsRunnable=83 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogInfo=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsNew=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapUsedM=325.47656 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTimedWaiting=119 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapMaxM=2200.0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Young Generation=11 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogError=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.ProcessName=NameNode 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogFatal=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillis=328 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemMaxM=2200.0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTerminated=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsWaiting=86 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.SessionId=null 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Context=jvm 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapUsedM=188.85118 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTotalExtraSleepTime=40 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Old Generation=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCount=11 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsBlocked=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogWarn=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsRunnable=83 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogInfo=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsNew=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapUsedM=325.47656 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTimedWaiting=119 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapMaxM=2200.0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Young Generation=11 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogError=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.ProcessName=DataNode 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogFatal=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillis=328 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,731 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_90th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_25th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_num_ops=14 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99.9th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_90th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_95th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_median=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_max=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_95th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_98th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_98th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_25th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:fsChecksumFailureCount=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_median=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_75th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_min=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Context=regionserver 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_max=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_min=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_90th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_75th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_max=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99.9th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_98th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_mean=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_95th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_num_ops=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_min=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_median=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99.9th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_mean=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_num_ops=186 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_25th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_mean=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_75th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99th_percentile=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context=dfs 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.1=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.2=dfs 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.2=8c5d72df62de 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.1=dfs 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.1=8c5d72df62de 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.2=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Context=master 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:numMasterWALs=0 2023-10-15T16:14:16,732 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,733 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsCount=0 2023-10-15T16:14:16,733 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_98th_percentile=0 2023-10-15T16:14:16,733 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_median=0 2023-10-15T16:14:16,733 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestRatePerSecond=0.0 2023-10-15T16:14:16,733 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerMissCount=0 2023-10-15T16:14:16,733 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_mean=0 2023-10-15T16:14:16,733 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99th_percentile=0 2023-10-15T16:14:16,733 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_median=0 2023-10-15T16:14:16,733 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_98th_percentile=0 2023-10-15T16:14:16,733 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_95th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_median=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_90th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_5min_rate=0.0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_min=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_95th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitRatio=0.0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_75th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_90th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_num_ops=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_min=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_25th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestCount=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedOutputBytes=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_90th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkHitCount=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadOpsInCurThread=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_median=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestRatePerSecond=0.0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_25th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_min=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_num_ops=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_num_ops=2 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_90th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockBytesScannedCount=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_max=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_75th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_max=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_mean=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_num_ops=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_25th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheEvictionCount=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99.9th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheCount=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:smallCompactionQueueLength=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCount=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_min=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_num_ops=6 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_mean=0 2023-10-15T16:14:16,734 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowIncrementCount=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushQueueLength=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_mean=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_75th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:localBytesRead=108773 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushCount=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_median=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutateFailedCount=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_max=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorPoolAllocationBytes=199680 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_98th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_min=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_median=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_min=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:updatesBlockedTime=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_median=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_mean=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitQueueLength=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_min=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_min=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_95th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCachingCount=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_25th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99.9th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileAge=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_median=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_75th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_max=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_95th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_num_ops=1 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_min=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cpRequestCount=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_90th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99th_percentile=0 2023-10-15T16:14:16,735 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_median=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_max=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_num_ops=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALCount=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_98th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitRatio=0.0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataMissCount=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestCount=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissCount=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_75th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_mean=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheEvictionCount=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheFreeSize=922070024 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_95th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_median=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_75th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_num_ops=1 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_max=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_median=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:activeScanners=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissCount=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_mean=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_max=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_max=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_95th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactionQueueLength=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_95th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_num_ops=1 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_max=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_1min_rate=0.0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_min=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_median=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_num_ops=0 2023-10-15T16:14:16,736 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_median=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_mean=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_75th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_min=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99.9th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_median=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_25th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_25th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_90th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_num_ops=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheSize=676856 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_num_ops=2 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_98th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_count=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_median=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowGetCount=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_median=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterRequestsCount=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_min=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_min=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_25th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_max=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileSize=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_min=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileCount=0 2023-10-15T16:14:16,737 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsCount=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99.9th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_98th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_num_ops=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedOutputBytes=5860 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_max=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_num_ops=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_max=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticBloomSize=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexMissCount=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseInfoThresholdExceeded=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCount=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCount=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_max=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedToMob=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedMemstoreBytes=928 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheExpressHitPercent=0.0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionServerStartTime=1697385851561 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedInputBytes=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedFromMob=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_max=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRequestCount=11 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_25th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcGetRequestCount=3 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_90th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeCount=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCountHitPercent=0.0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_max=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_mean=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_min=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_98th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_min=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsSize=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_mean=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowDeleteCount=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockedRequestCount=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_mean=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheAccessCount=0 2023-10-15T16:14:16,738 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_min=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_median=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_min=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_90th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_min=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_95th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_90th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheEvictedCount=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_90th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_max=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:avgStoreFileAge=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_25th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheCount=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitRequestCount=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_min=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalBytesRead=108773 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedToMob=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_max=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedInputBytes=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheSize=676856 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileCount=239 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_mean=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcScanRequestCount=2 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreOffHeapSize=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_75th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheMissCount=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowPutCount=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_max=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreSize=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_75th_percentile=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsSize=0 2023-10-15T16:14:16,739 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_mean=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_min=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_75th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_median=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_98th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_median=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Context=regionserver 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:scannerLeaseExpiredCount=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:minStoreFileAge=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitCount=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_98th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcFullScanRequestCount=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_mean=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitSuccessCount=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_mean=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_75th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_25th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCountPrimary=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_max=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataBlockCount=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_98th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_max=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_median=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsCount=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_mean=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsCount=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCachingCount=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFailedInsertionCount=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_90th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsSize=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean_rate=0.0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticIndexSize=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_min=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99.9th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissRatio=0.0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_75th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_max=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_mean=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:16,740 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_95th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMultiRequestCount=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheSize=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexHitCount=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_max=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_98th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedOutputBytes=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocal=0.0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_98th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_max=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_num_ops=3 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99.9th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_25th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_min=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_mean=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_15min_rate=0.0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99.9th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_min=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_95th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_75th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_median=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_median=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_mean=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_95th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitCount=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:largeCompactionQueueLength=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_mean=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutatePassedCount=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_max=0 2023-10-15T16:14:16,741 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_95th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissRatio=0.0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsCount=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_min=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_mean=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_min=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:shortCircuitBytesRead=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99.9th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_median=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_min=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoMissCount=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileIndexSize=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99.9th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_mean=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerHitCount=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_98th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsSize=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCountPrimary=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_min=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_max=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99.9th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_90th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_90th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_num_ops=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_max=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_mean=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_max=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_min=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRowActionRequestCount=0 2023-10-15T16:14:16,742 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMutateRequestCount=6 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheCount=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_25th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_mean=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataHitCount=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:zeroCopyBytesRead=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_98th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionCount=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_num_ops=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileCount=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_num_ops=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_median=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALSize=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_num_ops=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_mean=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_median=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheFreeSize=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:averageRegionSize=928 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_95th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaMissCount=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_75th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:filteredReadRequestCount=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaHitCount=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_75th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_num_ops=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_95th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_median=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCount=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_num_ops=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.serverName=8c5d72df62de,34443,1697385851561 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReads=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowAppendCount=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:numReferenceFiles=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedFromMob=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSize=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_max=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:16,743 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_90th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_median=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_mean=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_98th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadWins=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseWarnThresholdExceeded=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_max=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_25th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoHitCount=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_median=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreHeapSize=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_25th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_min=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99.9th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_25th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_num_ops=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_90th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_max=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99.9th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_max=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_median=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsSize=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_median=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_median=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_min=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkMissCount=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_95th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFreeSize=922070024 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheHitPercent=0.0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:16,744 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsCount=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_98th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_median=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestRatePerSecond=0.0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerMissCount=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_mean=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_median=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_98th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_95th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_median=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_90th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_5min_rate=0.0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_min=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_95th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitRatio=0.0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_75th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:16,745 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_90th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_num_ops=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_min=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_25th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestCount=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedOutputBytes=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_90th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkHitCount=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadOpsInCurThread=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_median=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestRatePerSecond=0.0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_25th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_min=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_num_ops=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_num_ops=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_90th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockBytesScannedCount=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_max=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_75th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_max=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_mean=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_num_ops=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_25th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheEvictionCount=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99.9th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheCount=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:smallCompactionQueueLength=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCount=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_min=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_num_ops=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_mean=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowIncrementCount=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushQueueLength=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_mean=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_75th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:localBytesRead=108773 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushCount=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_median=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutateFailedCount=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_max=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorPoolAllocationBytes=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_98th_percentile=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:16,746 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_min=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_median=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_min=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:updatesBlockedTime=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_median=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_mean=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitQueueLength=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_min=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_min=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_95th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCachingCount=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_25th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99.9th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileAge=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_median=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_75th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_max=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_95th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_num_ops=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_min=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cpRequestCount=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_90th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_median=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_max=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_num_ops=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALCount=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_98th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitRatio=0.0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataMissCount=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestCount=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissCount=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_75th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_mean=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheEvictionCount=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheFreeSize=922070024 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_95th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_median=0 2023-10-15T16:14:16,747 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_75th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_max=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_median=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:activeScanners=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissCount=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_mean=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_max=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_max=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_95th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactionQueueLength=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_95th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_num_ops=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_max=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_1min_rate=0.0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_min=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_median=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_num_ops=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_median=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_mean=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_75th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_min=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99.9th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_median=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_25th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_25th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_90th_percentile=0 2023-10-15T16:14:16,748 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_num_ops=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheSize=676856 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_num_ops=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_98th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_count=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_median=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowGetCount=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_median=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterRequestsCount=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_min=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_min=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_25th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_max=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileSize=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_min=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileCount=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsCount=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99.9th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_98th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_num_ops=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedOutputBytes=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_max=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_num_ops=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_max=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticBloomSize=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexMissCount=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseInfoThresholdExceeded=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCount=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCount=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_max=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedToMob=0 2023-10-15T16:14:16,749 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedMemstoreBytes=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheExpressHitPercent=0.0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionServerStartTime=1697385851811 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedInputBytes=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedFromMob=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_max=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRequestCount=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_25th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcGetRequestCount=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_90th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeCount=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCountHitPercent=0.0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_max=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_mean=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_min=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_98th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_min=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsSize=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_mean=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowDeleteCount=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockedRequestCount=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_mean=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheAccessCount=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_min=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_median=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_min=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_90th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_min=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_95th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_90th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheEvictedCount=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_90th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_max=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:avgStoreFileAge=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_25th_percentile=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheCount=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitRequestCount=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_min=0 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalBytesRead=108773 2023-10-15T16:14:16,750 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedToMob=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_max=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedInputBytes=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheSize=676856 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileCount=120 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_mean=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcScanRequestCount=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreOffHeapSize=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_75th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheMissCount=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowPutCount=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_max=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreSize=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_75th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsSize=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_mean=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_min=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_75th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_median=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_98th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_median=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Context=regionserver 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:scannerLeaseExpiredCount=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:minStoreFileAge=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitCount=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_98th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcFullScanRequestCount=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_mean=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitSuccessCount=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_mean=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_75th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_25th_percentile=0 2023-10-15T16:14:16,751 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCountPrimary=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_max=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataBlockCount=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_98th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_max=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_median=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsCount=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_mean=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsCount=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCachingCount=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFailedInsertionCount=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_90th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsSize=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean_rate=0.0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticIndexSize=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorUsedBufferCount=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_min=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99.9th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissRatio=0.0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_75th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_max=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_mean=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_95th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMultiRequestCount=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheSize=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexHitCount=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_max=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_98th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedOutputBytes=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocal=0.0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_98th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_max=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_num_ops=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99.9th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_25th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_min=0 2023-10-15T16:14:16,752 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_mean=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_15min_rate=0.0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99.9th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_min=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_95th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_75th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_median=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_median=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_mean=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_95th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitCount=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:largeCompactionQueueLength=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_mean=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutatePassedCount=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_max=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_95th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissRatio=0.0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsCount=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_min=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_mean=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_min=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:shortCircuitBytesRead=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99.9th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_median=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_min=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:16,753 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoMissCount=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileIndexSize=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99.9th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_mean=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerHitCount=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_98th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsSize=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCountPrimary=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_min=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_max=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99.9th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_90th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_90th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_num_ops=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_max=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_mean=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_max=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_min=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRowActionRequestCount=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMutateRequestCount=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheCount=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_25th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_mean=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataHitCount=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:zeroCopyBytesRead=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_98th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionCount=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_num_ops=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileCount=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_num_ops=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_median=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALSize=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_num_ops=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_mean=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_median=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheFreeSize=0 2023-10-15T16:14:16,754 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:averageRegionSize=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_95th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaMissCount=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_75th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:filteredReadRequestCount=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaHitCount=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_75th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_num_ops=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_95th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_median=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCount=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_num_ops=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.serverName=8c5d72df62de,40585,1697385851811 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReads=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowAppendCount=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:numReferenceFiles=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedFromMob=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSize=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_max=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_90th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_median=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_mean=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_98th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadWins=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseWarnThresholdExceeded=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_max=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_25th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoHitCount=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_median=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreHeapSize=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_25th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_min=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99.9th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_25th_percentile=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_num_ops=0 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:16,755 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_90th_percentile=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_max=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99.9th_percentile=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_max=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_median=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsSize=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_median=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_median=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_min=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkMissCount=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_95th_percentile=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFreeSize=922070024 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheHitPercent=0.0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:16,756 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsCount=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_98th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_median=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestRatePerSecond=0.0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerMissCount=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_mean=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_median=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_98th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_95th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_median=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_90th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_5min_rate=0.0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_min=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_95th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitRatio=0.0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_75th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_90th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_num_ops=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_min=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_25th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestCount=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedOutputBytes=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_90th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkHitCount=1 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadOpsInCurThread=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_median=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestRatePerSecond=0.0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_25th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_min=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_num_ops=0 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_num_ops=11 2023-10-15T16:14:16,757 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_90th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockBytesScannedCount=6840 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_max=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_75th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_max=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_mean=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_num_ops=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_25th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheEvictionCount=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99.9th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheCount=4 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:smallCompactionQueueLength=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCount=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_min=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_num_ops=7 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_num_ops=1 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_mean=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowIncrementCount=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushQueueLength=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_mean=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_75th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:localBytesRead=108773 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushCount=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_median=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutateFailedCount=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_max=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorPoolAllocationBytes=599040 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_98th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_min=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_median=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_min=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:updatesBlockedTime=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_median=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_mean=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitQueueLength=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_min=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_min=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_95th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCachingCount=4 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_25th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99.9th_percentile=0 2023-10-15T16:14:16,758 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileAge=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_median=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_75th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_max=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_95th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_num_ops=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_min=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cpRequestCount=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_90th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_median=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_max=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_num_ops=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALCount=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_98th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitRatio=0.72 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataMissCount=6 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestCount=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissCount=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_75th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_mean=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheEvictionCount=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheFreeSize=922061632 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_95th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_median=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_75th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_max=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_median=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:activeScanners=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissCount=7 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_mean=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_max=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_max=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_95th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:16,759 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactionQueueLength=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_95th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_num_ops=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_max=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_1min_rate=0.0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_min=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_median=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_num_ops=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_median=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_mean=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_75th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_min=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99.9th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_median=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_25th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_25th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_90th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_num_ops=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheSize=685248 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_num_ops=7 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_num_ops=11 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_98th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_count=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_median=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowGetCount=0 2023-10-15T16:14:16,760 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_median=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterRequestsCount=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_min=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_min=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_25th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_num_ops=1 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_max=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileSize=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_min=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileCount=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsCount=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99.9th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_98th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_num_ops=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedOutputBytes=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_max=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_num_ops=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_max=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticBloomSize=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexMissCount=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseInfoThresholdExceeded=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCount=4 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCount=7 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_max=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedToMob=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedMemstoreBytes=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheExpressHitPercent=81.81818181818183 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionServerStartTime=1697385888915 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedInputBytes=18105 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedFromMob=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_max=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRequestCount=21 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_25th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcGetRequestCount=1 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_90th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeCount=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,761 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCountHitPercent=72.0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_max=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_mean=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_min=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_98th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_min=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsSize=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_mean=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowDeleteCount=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockedRequestCount=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_mean=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheAccessCount=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_min=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_median=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_min=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_90th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_min=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_95th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_90th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheEvictedCount=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_90th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_max=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:avgStoreFileAge=12090 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_25th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheCount=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitRequestCount=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_num_ops=1 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_min=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalBytesRead=108773 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedToMob=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_max=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedInputBytes=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheSize=685248 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileCount=339 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_mean=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcScanRequestCount=11 2023-10-15T16:14:16,762 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreOffHeapSize=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_75th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheMissCount=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowPutCount=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_max=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreSize=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_75th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsSize=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_mean=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_min=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_75th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_median=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_98th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_median=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Context=regionserver 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:scannerLeaseExpiredCount=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:minStoreFileAge=2959 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitCount=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_98th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcFullScanRequestCount=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_mean=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitSuccessCount=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_mean=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_75th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_25th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCountPrimary=7 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_max=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataBlockCount=3 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_98th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_max=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_median=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsCount=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_mean=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsCount=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCachingCount=18 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFailedInsertionCount=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_90th_percentile=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsSize=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean_rate=0.0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticIndexSize=0 2023-10-15T16:14:16,763 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_min=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99.9th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissRatio=0.28 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_75th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_max=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_mean=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_95th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMultiRequestCount=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheSize=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexHitCount=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_max=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_98th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedOutputBytes=5921 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocal=0.0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_98th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_max=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_num_ops=1 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99.9th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_25th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_min=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_mean=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_15min_rate=0.0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99.9th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_min=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_95th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_75th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_median=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_median=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_mean=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_95th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitCount=18 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:largeCompactionQueueLength=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_mean=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutatePassedCount=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_max=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_95th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissRatio=0.0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsCount=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_min=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_mean=0 2023-10-15T16:14:16,764 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_min=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:shortCircuitBytesRead=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99.9th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_median=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_min=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoMissCount=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileIndexSize=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99.9th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_mean=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerHitCount=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_98th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsSize=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCountPrimary=18 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_min=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_max=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99.9th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_90th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_90th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_num_ops=1 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_max=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_mean=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_max=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_min=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRowActionRequestCount=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMutateRequestCount=8 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheCount=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_25th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationBytes=89913 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_mean=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataHitCount=17 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:zeroCopyBytesRead=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_98th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionCount=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_num_ops=1 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileCount=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_num_ops=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_median=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALSize=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_num_ops=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_mean=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_median=0 2023-10-15T16:14:16,765 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheFreeSize=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:averageRegionSize=32082 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_95th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaMissCount=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_75th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:filteredReadRequestCount=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaHitCount=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_75th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_num_ops=1 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_95th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_median=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCount=18 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_num_ops=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.serverName=8c5d72df62de,35419,1697385888915 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReads=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowAppendCount=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:numReferenceFiles=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedFromMob=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSize=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_max=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_90th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_median=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_mean=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_98th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadWins=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseWarnThresholdExceeded=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_max=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_25th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoHitCount=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_median=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreHeapSize=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_25th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_min=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99.9th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_25th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_num_ops=1 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_90th_percentile=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_max=0 2023-10-15T16:14:16,766 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99.9th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_max=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_median=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsSize=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_median=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_median=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_min=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkMissCount=1 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_95th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFreeSize=922061632 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheHitPercent=0.0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_98th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_num_ops=2 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.walReaderEditsBufferUsage=80117 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_25th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfHFileRefsQueue=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logReadInBytes=83847 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_75th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfLogQueue=-3 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_90th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99.9th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_num_ops=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedOps=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_min=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedHFiles=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_min=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_mean=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBytes=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_90th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedOps=1000 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedBatches=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_98th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_max=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_95th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedBatches=1 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_mean=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_95th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedHFiles=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_median=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.uncleanlyClosedLogs=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.closedLogsWithUnknownFileLength=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_max=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.repeatedLogFileBytes=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedRecoverQueues=5 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Context=regionserver 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedRecoverQueues=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.restartedLogReading=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.ignoredUncleanlyClosedLogContentsInBytes=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsRead=40 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.numInitializing=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99.9th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_75th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsFiltered=38 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_25th_percentile=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedLogs=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.failedBatches=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBatches=0 2023-10-15T16:14:16,767 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_median=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsAvgTime=0.0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpNumOps=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWrite=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosNumOps=26 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRemoved=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsNumOps=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksCached=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsAvgTime=94.0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalNumOps=9 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesRead=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpAvgTime=6326.928571428572 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesAvgTime=0.0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionTasks=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncCount=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:NumProcessedCommands=10 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpNumOps=27 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromLocalClient=28 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:VolumeFailures=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcInvalidReconstructionTasks=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosNumOps=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesRead=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksReadHits=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosAvgTime=561936.0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalReadTime=9 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesRead=52810 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsAvgTime=1.7142857142857142 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromRemoteClient=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksUncached=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DatanodeNetworkErrors=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromLocalClient=27 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksInPendingIBR=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockVerificationFailures=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosNumOps=54 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesWrite=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosNumOps=54 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.SessionId=null 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsNumOps=1 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesWritten=620489 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcFailedReconstructionTasks=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRead=27 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReplicated=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Context=dfs 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksVerified=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromRemoteClient=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksGetLocalPathInfo=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosAvgTime=20303.65 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpNumOps=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsNumOps=9 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksWritten=28 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsAvgTime=1.0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SumOfActorCommandQueueLength=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalWriteTime=12 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvicted=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcDecodingTimeNanos=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesNumOps=0 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosNumOps=1285 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpNumOps=28 2023-10-15T16:14:16,768 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosAvgTime=23635.714285714286 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesWritten=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosAvgTime=0.0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesWritten=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosAvgTime=10588.8 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpNumOps=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumFailedVolumes=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheCapacity=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheUsed=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:FailedStorageLocations=[Ljava.lang.String;@2b23d700 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksCached=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:LastVolumeFailureDate=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Capacity=1054589059072 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToUncache=7 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:DfsUsed=625022 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Remaining=941214948342 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToCache=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numRegionSizeReports=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_min=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_num_ops=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_98th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_num_ops=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numNamespaceInQuotaViolation=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_75th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_95th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_90th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_median=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_max=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.namespaceSpaceQuotaOverview=[] 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numTablesInQuotaViolation=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_num_ops=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99.9th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_max=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_98th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_median=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99th_percentile=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_min=0 2023-10-15T16:14:16,769 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_max=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Context=master 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99.9th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_95th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_median=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_mean=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_mean=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_max=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_25th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_mean=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_min=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_75th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_mean=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_90th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_min=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_median=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.tableSpaceQuotaOverview=[] 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numSpaceQuotas=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_25th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_num_ops=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenSubmittedCount=14 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignSubmittedCount=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_num_ops=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99.9th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_median=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_max=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_95th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_95th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_min=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_mean=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_min=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_median=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_min=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveSubmittedCount=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_25th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_mean=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99.9th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_max=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnFS=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerRegions=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_75th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_95th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignSubmittedCount=13 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_75th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_98th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_num_ops=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_90th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignFailedCount=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_num_ops=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_90th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignFailedCount=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_25th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_98th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_98th_percentile=0 2023-10-15T16:14:16,770 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveFailedCount=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_mean=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_75th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeFailedCount=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Context=master 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_num_ops=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseSubmittedCount=1 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:holes=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_75th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99.9th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_95th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_95th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeSubmittedCount=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99.9th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_median=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99.9th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_mean=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCount=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenSubmittedCount=1 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseFailedCount=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_max=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:inconsistentRegions=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_90th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenFailedCount=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnRS=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99.9th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_75th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_75th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_min=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_25th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerOpenRegions=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_95th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_25th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:overlaps=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenFailedCount=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_90th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_98th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitFailedCount=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_max=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_98th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_mean=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_num_ops=13 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_median=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_mean=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCountOverThreshold=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_min=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_75th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_90th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_num_ops=1 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_75th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_95th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_25th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_median=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_25th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitSubmittedCount=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_mean=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritOldestAge=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:emptyRegionInfoRegions=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_max=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_25th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99.9th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_90th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_min=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99.9th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_num_ops=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_max=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_95th_percentile=0 2023-10-15T16:14:16,771 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_median=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_max=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_90th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_25th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_25th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_min=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_90th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_95th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_max=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_num_ops=1 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_min=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:operationCount=15 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_98th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:deadServerOpenRegions=3 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_mean=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_median=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_min=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99.9th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_98th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_median=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_max=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_mean=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_num_ops=14 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_98th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_90th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_median=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_98th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_75th_percentile=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumFailedVolumes=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheUsed=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheCapacity=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksCached=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:LastVolumeFailureDate=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Capacity=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Context=FSDatasetState 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data2]'} 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:DfsUsed=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Remaining=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToCache=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToUnCache=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalDataFileIos=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalFileIoErrors=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateNumOps=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateNumOps=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateNumOps=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateNumOps=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateNumOps=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateNumOps=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Context=dfs 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalMetadataOperations=0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsNumOps=17 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportNumOps=14 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsNumOps=54 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedAvgTime=0.0 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoAvgTime=0.6666666666666666 2023-10-15T16:14:16,772 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoNumOps=675 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestNumOps=3 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedNumOps=159 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateNumOps=60 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.port=39027 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeNumOps=3 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportAvgTime=0.3333333333333333 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Context=rpcdetailed 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameNumOps=41 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportAvgTime=0.21428571428571427 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateAvgTime=5.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteNumOps=60 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingNumOps=206 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsNumOps=322 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportNumOps=3 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeAvgTime=0.6666666666666666 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionNumOps=1 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockNumOps=49 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteNumOps=33 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateNumOps=1 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesNumOps=16 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeNumOps=1 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatNumOps=600 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseNumOps=19 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseNumOps=5 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatAvgTime=0.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemMaxM=2200.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTerminated=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsWaiting=86 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.SessionId=null 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Context=jvm 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapUsedM=188.86484 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTotalExtraSleepTime=55 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Old Generation=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCount=11 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsBlocked=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogWarn=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsRunnable=83 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogInfo=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsNew=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapUsedM=327.47656 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTimedWaiting=119 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapMaxM=2200.0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Young Generation=11 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogError=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.ProcessName=DataNode 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogFatal=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillis=328 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDataNodeMessageCount=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastWrittenTransactionId=431 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:SnapshottableDirectories=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEncryptionZones=0 2023-10-15T16:14:16,773 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReconstructionBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastCheckpointTime=1697385856136 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ReadLockLongHoldCount=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumActiveClients=1 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastLogRoll=431 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumTimedOutPendingReconstructions=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:VolumeFailuresTotal=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ProvidedCapacityTotal=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingECBlockGroups=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedGB=0.0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecommissioningDataNodes=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.HAState=active 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyECBlockGroups=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:FilesTotal=109 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ScheduledReplicationBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureECBlockGroups=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptECBlockGroups=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MillisSinceLastLoadedEdits=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalECBlockGroups=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExcessBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalLoad=9 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExpiredHeartbeats=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInServiceLiveDataNodes=3 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumStaleStorages=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:WriteLockLongHoldCount=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlocksTotal=42 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:UnderReplicatedBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicatedBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsed=1391792 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalReplicatedBlocks=42 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:StaleDataNodes=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumFilesUnderConstruction=1 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemaining=2823754896354 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PostponedMisreplicatedBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedNonDFS=178447369040 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LockQueueLength=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptReplicatedBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Context=dfs 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomDeadDataNodes=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:Snapshots=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionECBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.TotalSyncTimes=11 5 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReplicationBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotal=3163767177216 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDeadDataNodes=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplOneBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalSyncCount=388 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastCheckpoint=431 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicationOneBlocks=0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumLiveDataNodes=3 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlockCapacity=4194304 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotalGB=2946.0 2023-10-15T16:14:16,774 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyBlocks=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomLiveDataNodes=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemainingGB=2630.0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:ReceivedBytes=15456 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcSlowCalls=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationSuccesses=46 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumDroppedConnections=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcClientBackoff=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationFailures=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumOpenConnections=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.port=35085 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:SentBytes=9822 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeNumOps=46 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationSuccesses=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:CallQueueLength=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Context=rpc 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationFailures=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeNumOps=46 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeNumOps=46 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:receivedBytes=288613 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationSuccesses=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_max=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFailures=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:14:16,775 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numOpenConnections=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_num_ops=1195 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_median=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationSuccesses=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_num_ops=1195 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_num_ops=1195 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_max=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_max=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_median=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.otherExceptions=5 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Context=regionserver 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_median=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_min=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_median=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_max=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_min=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_min=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveHandler=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_num_ops=1195 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_mean=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReadHandler=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_median=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_num_ops=1195 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_mean=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callDropped=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions=5 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFallbacks=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationFailures=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_min=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_max=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:queueSize=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveScanHandler=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:14:16,776 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_min=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:sentBytes=44933 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_mean=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_75th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_max=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_90th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_min=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_median=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_num_ops=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOnHeapSize=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_num_ops=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_num_ops=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_median=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_num_ops=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_75th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_min=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOffHeapSize=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_25th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_75th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_mean=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_25th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_25th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_mean=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tunerDoNothingCounter=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_25th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_num_ops=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:aboveHeapOccupancyLowWaterMarkCounter=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_98th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreSize=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_min=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_max=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_90th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_mean=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_75th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Context=regionserver 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99.9th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_mean=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_min=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockCacheSize=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_90th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockedFlushGauge=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_max=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_median=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_98th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:unblockedFlushGauge=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_max=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_max=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_num_ops=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_min=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_95th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_90th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_95th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_max=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_median=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_98th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_min=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:14:16,777 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99.9th_percentile=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_median=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_median=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99th_percentile=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99th_percentile=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_95th_percentile=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_mean=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_95th_percentile=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_98th_percentile=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalDataFileIos=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalFileIoErrors=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateNumOps=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateNumOps=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateNumOps=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateNumOps=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateNumOps=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateNumOps=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Context=dfs 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalMetadataOperations=0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:HttpPort=null 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XceiverCount=3 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"2","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"599"}] 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:ClusterId=testClusterID 2023-10-15T16:14:16,778 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3":{"numBlocks":21,"usedSpace":108926,"freeSpace":470674169856,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4":{"numBlocks":21,"usedSpace":516096,"freeSpace":470540196854,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":133973002}} 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DatanodeHostname=127.0.0.1 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SecurityEnabled=false 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:RpcPort=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DataPort=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SoftwareVersion=3.2.4 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SlowDisks=null 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XmitsInProgress=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DNStartedTimeInMillis=1697385857021 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsBatchedInSync=44 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsNumOps=387 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsAvgTime=0.11764705882352941 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportNumOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeNumOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteSnapshotOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsAvgTime=0.09523809523809523 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AllowSnapshotOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:RenameSnapshotOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeAvgTime=0.08333333333333333 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesCreated=168 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesAppended=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportAvgTime=0.0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageAvgTime=0.0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeNumOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeNumOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeNumOps=126 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeAvgTime=0.0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumTimesReReplicationNotScheduled=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SafeModeTime=1 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalNumOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetListingOps=132 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.ProcessName=NameNode 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageNumOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TotalFileOps=1303 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSymlinkOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AddBlockOps=49 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DisallowSnapshotOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteFileOps=23 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsNumOps=431 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSnapshotOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PendingEditsCount=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ListSnapshottableDirOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockReceivedAndDeletedOps=159 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.SessionId=null 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditNumOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FsImageLoadTime=23 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateFileOps=60 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Context=dfs 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesRenamed=41 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TimeoutReReplications=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsQueued=1 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedNumOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageNumOps=0 2023-10-15T16:14:16,779 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesInGetListingOps=146 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FileInfoOps=675 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetLinkTargetOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetAdditionalDatanodeOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsBatched=68 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesDeleted=39 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SuccessfulReReplications=21 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetBlockLocations=322 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesTruncated=1 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SnapshotDiffReportOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportNumOps=6 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeNumOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportAvgTime=0.3333333333333333 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumFailedVolumes=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheUsed=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheCapacity=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksCached=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:LastVolumeFailureDate=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Capacity=1054589059072 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Context=FSDatasetState 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:DfsUsed=383910 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Remaining=941226220534 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToCache=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToUnCache=7 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalDataFileIos=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateNumOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalFileIoErrors=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateNumOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateNumOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateNumOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateNumOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateNumOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateNumOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateNumOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateNumOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Context=dfs 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalMetadataOperations=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateAvgTime=0.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessNumOps=1 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureNumOps=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailures=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsNumOps=2 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailuresTotal=0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Context=ugi 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsAvgTime=6.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessAvgTime=10.0 2023-10-15T16:14:16,780 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureAvgTime=0.0 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfSnapshottableDirs=0 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheCapacity=0 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheUsed=0 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ProvidedCapacity=0 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DistinctVersionCount=1 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocksWithReplicationFactorOne=0 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentRemaining=89.25292 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentUsed=4.3991608E-5 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:TotalBlocks=42 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NonDfsUsedSpace=178447369040 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Safemode= 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:UpgradeFinalized=true 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:EnteringMaintenanceNodes={} 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentBlockPoolUsed=4.3991608E-5 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:LiveNodes={"127.0.0.1:40557":{"infoAddr":"127.0.0.1:35357","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:40557","lastContact":2,"usedSpace":625022,"adminState":"In Service","nonDfsUsedSpace":59479094914,"capacity":1054589059072,"numBlocks":41,"version":"3.2.4","used":625022,"remaining":941254826998,"blockScheduled":1,"blockPoolUsed":625022,"blockPoolUsedPercent":5.9266877E-5,"volfails":0,"lastBlockReport":9},"127.0.0.1:46365":{"infoAddr":"127.0.0.1:35919","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46365","lastContact":2,"usedSpace":383910,"adminState":"In Service","nonDfsUsedSpace":59477673050,"capacity":1054589059072,"numBlocks":41,"version":"3.2.4","used":383910,"remaining":941256494070,"blockScheduled":1,"blockPoolUsed":383910,"blockPoolUsedPercent":3.6403755E-5,"volfails":0,"lastBlockReport":9},"127.0.0.1:46399":{"infoAddr":"127.0.0.1:44887","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46399","lastContact":1,"usedSpace":382860,"adminState":"In Service","nonDfsUsedSpace":59490601076,"capacity":1054589059072,"numBlocks":41,"version":"3.2.4","used":382860,"remaining":941243575286,"blockScheduled":1,"blockPoolUsed":382860,"blockPoolUsedPercent":3.6304187E-5,"volfails":0,"lastBlockReport":9}} 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CompileInfo=2022-07-12T11:58Z by ubuntu from branch-3.2.4 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolId=BP-1531809096-172.17.0.3-1697385856100 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ClusterId=testClusterID 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DeadNodes={} 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:JournalTransactionInfo={"MostRecentCheckpointTxId":"0","LastAppliedOrWrittenTxId":"431"} 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Free=2823754896354 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolUsedSpace=1391792 2023-10-15T16:14:16,782 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Threads=288 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocks=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirStatuses={"active":{"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":"IMAGE_AND_EDITS","/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":"IMAGE_AND_EDITS"},"failed":{}} 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:SoftwareVersion=3.2.4 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:VerifyECWithTopologyResult={"isSupported":"false","resultMessage":"The number of DataNodes (3) is less than the minimum required number of DataNodes (9) for the erasure coding policies: RS-6-3-1024k"} 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Used=1391792 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Total=3163767177216 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NodeUsage={"nodeUsage":{"min":"0.00%","median":"0.00%","max":"0.00%","stdDev":"0.00%"}} 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NNStartedTimeInMillis=1697385856352 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DecomNodes={} 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CorruptFiles=[] 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirSize={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":737,"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":737} 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameJournalStatus=[{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"},{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"}] 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemMaxM=2200.0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTerminated=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsWaiting=86 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.SessionId=null 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Context=jvm 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapUsedM=188.87222 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapMaxM=-1.0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTotalExtraSleepTime=36 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Hostname=8c5d72df62de 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Old Generation=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCount=11 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsBlocked=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogWarn=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapCommittedM=1000.0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsRunnable=83 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogInfo=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsNew=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapUsedM=328.47656 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTimedWaiting=119 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapMaxM=2200.0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Young Generation=11 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogError=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.ProcessName=DataNode 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogFatal=0 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillis=328 2023-10-15T16:14:16,783 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlocksTotal=42 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TopUserOpCounts={"timestamp":"2023-10-15T16:14:16+0000","windows":[{"ops":[{"opType":"*","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"getfileinfo","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]}],"windowLenMs":60000},{"ops":[{"opType":"*","totalCount":4,"allUsers":[{"user":"jenkins","count":4}],"topUsers":[{"user":"jenkins","count":4}]},{"opType":"listStatus","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"getfileinfo","totalCount":3,"allUsers":[{"user":"jenkins","count":3}],"topUsers":[{"user":"jenkins","count":3}]}],"windowLenMs":300000},{"ops":[{"opType":"*","totalCount":1424,"allUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":300},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":300},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}]},{"opType":"datanodeReport","totalCount":14,"allUsers":[{"user":"jenkins","count":14}],"topUsers":[{"user":"jenkins","count":14}]},{"opType":"listStatus","totalCount":206,"allUsers":[{"user":"jenkins","count":66},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}],"topUsers":[{"user":"jenkins","count":66},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}]},{"opType":"safemode_get","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"delete","totalCount":33,"allUsers":[{"user":"jenkins.hfs.4","count":13},{"user":"jenkins","count":12},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}],"topUsers":[{"user":"jenkins.hfs.4","count":13},{"user":"jenkins","count":12},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}]},{"opType":"getfileinfo","totalCount":675,"allUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":136},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}],"topUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":136},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}]},{"opType":"truncate","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"rename","totalCount":41,"allUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":12},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":12},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}]},{"opType":"mkdirs","totalCount":54,"allUsers":[{"user":"jenkins","count":32},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}],"topUsers":[{"user":"jenkins","count":32},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}]},{"opType":"create","totalCount":60,"allUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":17},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":17},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}]},{"opType":"setPermission","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"setTimes","totalCount":16,"allUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}]},{"opType":"open","totalCount":322,"allUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":4}],"topUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":4}]}],"windowLenMs":1500000}]} 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:SnapshotStats={"SnapshottableDirectories":0,"Snapshots":0} 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:UnderReplicatedBlocks=0 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityUsed=1391792 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEncryptionZones=0 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityRemaining=2823754896354 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReconstructionBlocks=0 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomDeadDataNodes=0 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FSState=Operational 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:VolumeFailuresTotal=0 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ProvidedCapacityTotal=0 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:16,786 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecommissioningDataNodes=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReplicationBlocks=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncTimes=11 5 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingDeletionBlocks=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityTotal=3163767177216 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FilesTotal=109 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ScheduledReplicationBlocks=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDeadDataNodes=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncCount=388 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FsLockQueueLength=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:MaxObjects=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalLoad=9 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleDataNodes=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumLiveDataNodes=3 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlockDeletionStartTime=1697385856352 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInServiceLiveDataNodes=3 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:LowRedundancyBlocks=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomLiveDataNodes=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleStorages=0 2023-10-15T16:14:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(109): << End DataNode metrics dump 2023-10-15T16:14:17,018 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(83): >> Begin DataNode metrics dump 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_num_ops=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_mean=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_num_ops=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_25th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_95th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_90th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_90th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_max=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_98th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_25th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_75th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_min=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_75th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_min=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_max=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_max=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_25th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99.9th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_75th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_mean=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99.9th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_75th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_median=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_95th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_98th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99.9th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_median=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_25th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Context=master 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_median=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_95th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_98th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_90th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_mean=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99.9th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_95th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_min=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_max=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_98th_percentile=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_mean=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_num_ops=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_min=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_median=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_num_ops=0 2023-10-15T16:14:17,019 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_90th_percentile=0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:HttpPort=null 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XceiverCount=3 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"0","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"600"}] 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:ClusterId=testClusterID 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1":{"numBlocks":21,"usedSpace":266240,"freeSpace":470668120064,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2":{"numBlocks":21,"usedSpace":117670,"freeSpace":470534147062,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":133973002}} 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DatanodeHostname=127.0.0.1 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SecurityEnabled=false 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:RpcPort=0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DataPort=0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SoftwareVersion=3.2.4 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SlowDisks=null 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XmitsInProgress=0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DNStartedTimeInMillis=1697385856691 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Context=regionserver 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.port=44717 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Context=rpcdetailed 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthNumOps=50 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpNumOps=4 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWrite=0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosNumOps=44 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRemoved=7 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalAvgTime=0.9795918367346939 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsNumOps=0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksCached=0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsAvgTime=2.0 2023-10-15T16:14:17,020 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DataNodeActiveXceiversCount=1 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalNumOps=200 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsNumOps=53 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesRead=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpAvgTime=3792.75 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesAvgTime=0.0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionTasks=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncCount=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:NumProcessedCommands=201 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpNumOps=92 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromLocalClient=48 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:VolumeFailures=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosNumOps=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesRead=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosAvgTime=1433727.0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalReadTime=16 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesRead=298317 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsAvgTime=1.0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromRemoteClient=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksUncached=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DatanodeNetworkErrors=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromLocalClient=92 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksInPendingIBR=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockVerificationFailures=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosNumOps=184 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesWrite=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosNumOps=184 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.SessionId=null 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsNumOps=1 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesWritten=973747 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRead=92 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReplicated=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Context=dfs 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksVerified=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromRemoteClient=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosAvgTime=20216.833333333332 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsNumOps=200 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksWritten=48 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsAvgTime=0.9183673469387755 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalWriteTime=21 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcDecodingTimeNanos=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesNumOps=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosNumOps=1700 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpNumOps=48 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosAvgTime=5276.454545454545 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesWritten=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosAvgTime=12045.5 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpNumOps=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumFailedVolumes=0 2023-10-15T16:14:17,021 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheUsed=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheCapacity=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksCached=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:LastVolumeFailureDate=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Capacity=1054589059072 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Context=FSDatasetState 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:DfsUsed=625022 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Remaining=941225311222 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToCache=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToUnCache=7 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheCleared=1 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheHit=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Context=rpc 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheUpdated=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalDataFileIos=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalFileIoErrors=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateNumOps=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateNumOps=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateNumOps=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateNumOps=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateNumOps=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateNumOps=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Context=dfs 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalMetadataOperations=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:MissingECBlockGroups=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:LowRedundancyECBlockGroups=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:BytesInFutureECBlockGroups=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:CorruptECBlockGroups=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:PendingDeletionECBlocks=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:TotalECBlockGroups=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheCleared=1 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheHit=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Context=rpc 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheUpdated=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:LastHATransitionTime=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:HostAndPort=localhost:39027 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:State=active 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowPeersReport=null 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowDisksReport=null 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:NNRole=NameNode 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:BytesWithFutureGenerationStamps=0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SecurityEnabled=false 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.port=35753 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Context=rpcdetailed 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthNumOps=55 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,022 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:ReceivedBytes=18480 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcSlowCalls=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationSuccesses=55 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumDroppedConnections=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcClientBackoff=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationFailures=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumOpenConnections=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.port=35753 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:SentBytes=11751 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeNumOps=55 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationSuccesses=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:CallQueueLength=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Context=rpc 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationFailures=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeNumOps=55 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeNumOps=55 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:ReceivedBytes=16800 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcSlowCalls=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationSuccesses=50 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumDroppedConnections=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcClientBackoff=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationFailures=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumOpenConnections=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.port=44717 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:SentBytes=10673 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeNumOps=50 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationSuccesses=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:CallQueueLength=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Context=rpc 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationFailures=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeNumOps=50 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeNumOps=50 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemMaxM=2200.0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTerminated=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsWaiting=86 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.SessionId=null 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Context=jvm 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapUsedM=188.82245 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTotalExtraSleepTime=51 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Old Generation=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCount=11 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsBlocked=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogWarn=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsRunnable=83 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogInfo=0 2023-10-15T16:14:17,023 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsNew=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapUsedM=323.47656 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTimedWaiting=119 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapMaxM=2200.0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Young Generation=11 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogError=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.ProcessName=NameNode 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogFatal=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillis=328 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumFailedVolumes=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheUsed=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheCapacity=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksCached=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:LastVolumeFailureDate=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Capacity=1054589059072 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Context=FSDatasetState 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:DfsUsed=382860 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Remaining=941225311222 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToCache=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToUnCache=7 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemMaxM=2200.0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTerminated=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsWaiting=86 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.SessionId=null 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Context=jvm 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapUsedM=188.82245 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTotalExtraSleepTime=22 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Old Generation=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCount=11 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsBlocked=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogWarn=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsRunnable=84 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogInfo=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsNew=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapUsedM=323.47656 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTimedWaiting=118 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapMaxM=2200.0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Young Generation=11 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogError=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.ProcessName=DataNode 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogFatal=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillis=328 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_num_ops=1891 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99th_percentile=1 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_max=4 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_mean=206 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:appendCount=1891 2023-10-15T16:14:17,024 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:lowReplicaRollRequest=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99th_percentile=332 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99.9th_percentile=4 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:sizeRollRequest=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendSize=297021 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_max=1 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowAppendCount=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:successfulLogRolls=24 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99.9th_percentile=333 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_TimeRangeCount_0-1=11 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_25th_percentile=36 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendSize=9557 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_90th_percentile=1 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendCount=1011 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_25th_percentile=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_10-100=4 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99th_percentile=4 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_98th_percentile=4 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_75th_percentile=298 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_75th_percentile=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_max=334 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_95th_percentile=4 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_mean=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendCount=48 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_100-1000=7 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:errorRollRequest=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:rollRequest=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_num_ops=1804 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_median=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_3-10=1 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_98th_percentile=1 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_min=36 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_min=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_median=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Context=regionserver 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_0-1=10 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendCount=45 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_95th_percentile=326 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendSize=74104 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_median=262 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_98th_percentile=331 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_25th_percentile=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_75th_percentile=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_mean=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99.9th_percentile=1 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendSize=14494 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:writtenBytes=395176 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_90th_percentile=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_95th_percentile=1 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowSyncRollRequest=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_num_ops=1891 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_90th_percentile=319 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_min=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendCount=787 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicationOneBlocks=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:CorruptReplicatedBlocks=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicatedBlocks=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:TotalReplicatedBlocks=42 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalDataFileIos=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalFileIoErrors=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateNumOps=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateNumOps=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateNumOps=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateNumOps=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateNumOps=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateNumOps=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Context=dfs 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalMetadataOperations=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotNumOps=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishNumOps=0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotAvgTime=0.0 2023-10-15T16:14:17,025 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Context=metricssystem 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSources=67 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:DroppedPubAll=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishAvgTime=0.0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSources=67 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSinks=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSinks=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheCleared=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheHit=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Context=rpc 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheUpdated=134 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWrite=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosNumOps=28 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRemoved=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsNumOps=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksCached=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsAvgTime=3.0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalNumOps=9 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesRead=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpAvgTime=9.0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpAvgTime=6810.7692307692305 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesAvgTime=0.0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionTasks=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncCount=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:NumProcessedCommands=10 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpNumOps=31 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromLocalClient=28 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:VolumeFailures=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosNumOps=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesRead=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosAvgTime=554484.75 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalReadTime=7 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesRead=57958 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsAvgTime=1.5714285714285714 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromRemoteClient=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksUncached=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DatanodeNetworkErrors=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromLocalClient=31 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksInPendingIBR=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockVerificationFailures=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosNumOps=62 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesWrite=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosNumOps=62 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.SessionId=null 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsNumOps=1 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesWritten=620489 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRead=31 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReplicated=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Context=dfs 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksVerified=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromRemoteClient=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosAvgTime=19658.18181818182 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsNumOps=9 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,026 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksWritten=28 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsAvgTime=1.0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalWriteTime=12 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcDecodingTimeNanos=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesNumOps=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosNumOps=1285 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpNumOps=28 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosAvgTime=4301.205882352941 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesWritten=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosAvgTime=9971.454545454546 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpAvgTime=0.8181818181818182 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpNumOps=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemMaxM=2200.0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTerminated=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsWaiting=86 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.SessionId=null 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Context=jvm 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapUsedM=188.83234 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTotalExtraSleepTime=62 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Old Generation=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCount=11 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsBlocked=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogWarn=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsRunnable=83 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogInfo=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsNew=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapUsedM=323.47656 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTimedWaiting=119 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapMaxM=2200.0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Young Generation=11 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogError=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.ProcessName=DataNode 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogFatal=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillis=328 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Context=regionserver 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:numTables=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99.9th_percentile=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_90th_percentile=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_95th_percentile=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterRequests=1101 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99th_percentile=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterWriteRequests=1059 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_mean=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterReadRequests=74 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_min=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_25th_percentile=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashSubmittedCount=4 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashFailedCount=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_98th_percentile=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Context=master 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_num_ops=2 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_75th_percentile=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_max=0 2023-10-15T16:14:17,027 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_median=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:receivedBytes=6048147 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationSuccesses=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_max=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFailures=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numOpenConnections=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_num_ops=1266 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_median=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationSuccesses=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_num_ops=1266 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_num_ops=1266 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_max=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_max=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_median=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.otherExceptions=3 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Context=master 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_median=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_min=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_median=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_max=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_min=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_min=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveHandler=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:14:17,028 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_num_ops=1266 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_mean=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReadHandler=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_median=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_num_ops=1266 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_mean=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callDropped=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions=3 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFallbacks=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationFailures=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_min=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_max=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:queueSize=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveScanHandler=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_min=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:sentBytes=16055 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumFailedVolumes=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheUsed=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheCapacity=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksCached=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:LastVolumeFailureDate=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Capacity=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Context=FSDatasetState 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data4]'} 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:DfsUsed=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Remaining=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToCache=0 2023-10-15T16:14:17,029 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToUnCache=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:HttpPort=null 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XceiverCount=3 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"1","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"598"}] 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:ClusterId=testClusterID 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5":{"numBlocks":21,"usedSpace":266240,"freeSpace":470667722752,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6":{"numBlocks":21,"usedSpace":116620,"freeSpace":470533749750,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":133973002}} 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DatanodeHostname=127.0.0.1 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SecurityEnabled=false 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:RpcPort=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DataPort=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SoftwareVersion=3.2.4 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SlowDisks=null 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XmitsInProgress=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DNStartedTimeInMillis=1697385857417 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalDataFileIos=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalFileIoErrors=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateNumOps=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateNumOps=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateNumOps=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateNumOps=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateNumOps=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateNumOps=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Context=dfs 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalMetadataOperations=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumFailedVolumes=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheUsed=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheCapacity=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksCached=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:LastVolumeFailureDate=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Capacity=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Context=FSDatasetState 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data6]'} 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:DfsUsed=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Remaining=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToCache=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToUnCache=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:ReceivedBytes=744936 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeAvgTime=0.009771986970684038 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcSlowCalls=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationSuccesses=27 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumDroppedConnections=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.NumOpenConnectionsPerUser={"jenkins":1} 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcClientBackoff=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeAvgTime=0.006514657980456026 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationFailures=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumOpenConnections=1 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.port=39027 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:SentBytes=627674 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeNumOps=2342 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationSuccesses=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:CallQueueLength=0 2023-10-15T16:14:17,030 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Context=rpc 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationFailures=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeNumOps=2342 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.serverName=ClientNamenodeProtocol 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeNumOps=2342 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.port=35085 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Context=rpcdetailed 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthNumOps=46 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemMaxM=2200.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTerminated=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsWaiting=86 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.SessionId=null 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Context=jvm 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapUsedM=188.845 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTotalExtraSleepTime=30 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Old Generation=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCount=11 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsBlocked=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogWarn=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsRunnable=83 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogInfo=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsNew=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapUsedM=323.47656 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTimedWaiting=119 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapMaxM=2200.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Young Generation=11 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogError=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.ProcessName=DataNode 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogFatal=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillis=328 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsBatchedInSync=31 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsNumOps=224 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsAvgTime=0.13043478260869565 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportNumOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeNumOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteSnapshotOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsAvgTime=0.03896103896103896 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AllowSnapshotOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:RenameSnapshotOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeAvgTime=0.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesCreated=80 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesAppended=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportAvgTime=0.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageAvgTime=0.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeNumOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeNumOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeNumOps=3 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeAvgTime=0.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumTimesReReplicationNotScheduled=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SafeModeTime=3 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalNumOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetListingOps=66 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.ProcessName=NameNode 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageNumOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TotalFileOps=540 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSymlinkOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AddBlockOps=28 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DisallowSnapshotOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteFileOps=5 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsNumOps=255 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSnapshotOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PendingEditsCount=1 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ListSnapshottableDirOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockReceivedAndDeletedOps=96 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.SessionId=null 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditNumOps=0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FsImageLoadTime=173 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateFileOps=36 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Context=dfs 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesRenamed=17 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:14:17,031 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TimeoutReReplications=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsQueued=1 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedNumOps=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageNumOps=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageAvgTime=0.0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesInGetListingOps=35 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FileInfoOps=270 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetLinkTargetOps=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetAdditionalDatanodeOps=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsBatched=9 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesDeleted=1 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SuccessfulReReplications=2 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetBlockLocations=118 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesTruncated=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SnapshotDiffReportOps=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportNumOps=6 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeNumOps=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditAvgTime=0.0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportAvgTime=0.5 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumFailedVolumes=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheCapacity=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheUsed=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:FailedStorageLocations=[Ljava.lang.String;@1db6f600 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksCached=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:LastVolumeFailureDate=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Capacity=1054589059072 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToUncache=7 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:DfsUsed=383910 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Remaining=941201333238 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToCache=0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpNumOps=5 2023-10-15T16:14:17,032 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWrite=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosNumOps=48 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRemoved=7 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalAvgTime=1.0303030303030303 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsNumOps=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksCached=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsAvgTime=2.0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DataNodeActiveXceiversCount=1 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalNumOps=200 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsNumOps=51 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesRead=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpAvgTime=3792.5 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesAvgTime=0.0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionTasks=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncCount=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:NumProcessedCommands=201 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpNumOps=99 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromLocalClient=48 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:VolumeFailures=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosNumOps=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesRead=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosAvgTime=388889.5 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalReadTime=29 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesRead=688752 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsAvgTime=0.75 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromRemoteClient=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksUncached=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DatanodeNetworkErrors=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromLocalClient=99 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksInPendingIBR=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockVerificationFailures=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosNumOps=200 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesWrite=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosNumOps=200 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.SessionId=null 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsNumOps=1 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesWritten=973747 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRead=99 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReplicated=0 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Context=dfs 2023-10-15T16:14:17,033 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksVerified=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromRemoteClient=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosAvgTime=45642.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsNumOps=200 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksWritten=48 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsAvgTime=0.9797979797979798 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalWriteTime=22 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcDecodingTimeNanos=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesNumOps=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosNumOps=1700 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpNumOps=48 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosAvgTime=4914.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesWritten=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosAvgTime=9294.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpNumOps=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWrite=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosNumOps=30 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRemoved=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsNumOps=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksCached=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsAvgTime=95.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalNumOps=9 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesRead=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpAvgTime=6809.0 2023-10-15T16:14:17,034 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesAvgTime=0.0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionTasks=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncCount=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:NumProcessedCommands=10 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpNumOps=35 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromLocalClient=28 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:VolumeFailures=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosNumOps=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesRead=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosAvgTime=352052.375 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalReadTime=16 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesRead=75811 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsAvgTime=1.3846153846153846 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromRemoteClient=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksUncached=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DatanodeNetworkErrors=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromLocalClient=35 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksInPendingIBR=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockVerificationFailures=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosNumOps=70 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesWrite=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosNumOps=70 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.SessionId=null 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsNumOps=1 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesWritten=620489 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRead=35 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReplicated=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Context=dfs 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksVerified=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromRemoteClient=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosAvgTime=25843.916666666668 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsNumOps=9 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksWritten=28 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsAvgTime=1.0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalWriteTime=12 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcDecodingTimeNanos=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesNumOps=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosNumOps=1285 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpNumOps=28 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosAvgTime=4740.882352941177 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesWritten=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,035 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosAvgTime=9895.041666666666 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpAvgTime=0.75 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpNumOps=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpNumOps=4 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWrite=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosNumOps=32 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRemoved=7 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalAvgTime=0.9795918367346939 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsNumOps=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksCached=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsAvgTime=2.0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DataNodeActiveXceiversCount=1 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalNumOps=200 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsNumOps=55 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesRead=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpAvgTime=3793.25 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesAvgTime=0.0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionTasks=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncCount=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:NumProcessedCommands=201 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpNumOps=89 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromLocalClient=48 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:VolumeFailures=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosNumOps=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesRead=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosAvgTime=304427.5 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalReadTime=12 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesRead=339498 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsAvgTime=0.75 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromRemoteClient=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksUncached=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DatanodeNetworkErrors=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromLocalClient=89 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksInPendingIBR=0 2023-10-15T16:14:17,036 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockVerificationFailures=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosNumOps=178 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesWrite=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosNumOps=178 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.SessionId=null 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsNumOps=1 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesWritten=973747 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRead=89 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReplicated=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Context=dfs 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksVerified=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromRemoteClient=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosAvgTime=17297.333333333332 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsNumOps=200 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksWritten=48 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsAvgTime=0.9489795918367347 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalWriteTime=28 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcDecodingTimeNanos=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesNumOps=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosNumOps=1700 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpNumOps=48 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosAvgTime=4986.545454545455 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesWritten=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosAvgTime=11669.166666666666 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpAvgTime=0.3333333333333333 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpNumOps=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDataNodeMessageCount=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastWrittenTransactionId=255 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:SnapshottableDirectories=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEncryptionZones=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReconstructionBlocks=0 2023-10-15T16:14:17,037 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastCheckpointTime=1697385845476 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ReadLockLongHoldCount=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumActiveClients=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastLogRoll=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumTimedOutPendingReconstructions=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:VolumeFailuresTotal=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ProvidedCapacityTotal=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingECBlockGroups=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedGB=0.0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecommissioningDataNodes=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.HAState=active 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyECBlockGroups=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:FilesTotal=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ScheduledReplicationBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureECBlockGroups=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptECBlockGroups=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MillisSinceLastLoadedEdits=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalECBlockGroups=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExcessBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalLoad=32 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExpiredHeartbeats=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInServiceLiveDataNodes=3 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumStaleStorages=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:WriteLockLongHoldCount=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlocksTotal=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:UnderReplicatedBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicatedBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsed=495713 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalReplicatedBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:StaleDataNodes=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumFilesUnderConstruction=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemaining=2816043276636 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PostponedMisreplicatedBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedNonDFS=185085906847 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LockQueueLength=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptReplicatedBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Context=dfs 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomDeadDataNodes=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:Snapshots=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionECBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.TotalSyncTimes= 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReplicationBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotal=3163767177216 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDeadDataNodes=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplOneBlocks=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalSyncCount=0 2023-10-15T16:14:17,038 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastCheckpoint=255 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicationOneBlocks=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumLiveDataNodes=3 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlockCapacity=4194304 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotalGB=2946.0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyBlocks=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomLiveDataNodes=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemainingGB=2623.0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Context=regionserver 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheCleared=1 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheHit=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Context=rpc 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheUpdated=65 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemMaxM=2200.0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTerminated=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsWaiting=86 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.SessionId= 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Context=jvm 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapUsedM=188.84634 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Old Generation=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCount=11 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsBlocked=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogWarn=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsRunnable=83 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogInfo=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsNew=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapUsedM=324.47656 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTimedWaiting=119 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapMaxM=2200.0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Young Generation=11 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogError=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.ProcessName=Master 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogFatal=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillis=328 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsCount=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeTotal=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointPercentComplete=1.0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageTotal=1 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsTotal=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointCount=0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:ElapsedTime=681 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsPercentComplete=1.0 2023-10-15T16:14:17,039 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsElapsedTime=7 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointElapsedTime=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModePercentComplete=1.0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:PercentComplete=1.0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointTotal=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeCount=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeElapsedTime=13 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageElapsedTime=62 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageCount=1 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImagePercentComplete=1.0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumFailedVolumes=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheCapacity=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheUsed=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:FailedStorageLocations=[Ljava.lang.String;@13711bc6 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksCached=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:LastVolumeFailureDate=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Capacity=1054589059072 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToUncache=7 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:DfsUsed=382860 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Remaining=941200800758 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToCache=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsCount=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeTotal=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointPercentComplete=1.0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageTotal=1 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsTotal=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointCount=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:ElapsedTime=681 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsPercentComplete=1.0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsElapsedTime=7 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointElapsedTime=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModePercentComplete=1.0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:PercentComplete=1.0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointTotal=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeCount=0 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeElapsedTime=13 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageElapsedTime=62 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageCount=1 2023-10-15T16:14:17,040 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImagePercentComplete=1.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RackLocalityCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionCountSkewCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaHostCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ServerLocalityCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MemStoreSizeCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_95th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_Overall=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ReadRequestCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_25th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_98th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_min=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Context=master 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_WriteRequestCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaRackCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_PrimaryRegionCountSkewCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:miscInvocationCount=18 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_median=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_num_ops=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_75th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MoveCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_max=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_mean=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_CPRequestCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_TableSkewCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_StoreFileCostFunction=0.0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_90th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.isBalancerActive=true 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99.9th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_95th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99.9th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_median=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_mean=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_25th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_max=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_95th_percentile=0 2023-10-15T16:14:17,041 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_num_ops=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_25th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_98th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_min=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_mean=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_90th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_75th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_90th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_mean=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_num_ops=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_95th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_min=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_median=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_num_ops=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_mean=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_98th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_75th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_median=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99.9th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Context=regionserver 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_max=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_num_ops=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_num_ops=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_min=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_90th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_mean=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_95th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_max=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_75th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_75th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_min=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_num_ops=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_75th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_max=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_min=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_98th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_90th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_max=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_98th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_90th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_75th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_95th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,042 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_25th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_max=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_max=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_98th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_num_ops=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99.9th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_min=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_98th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_95th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_min=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_median=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:BlockBytesScannedCount=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99.9th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_25th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_median=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_90th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_95th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_num_ops=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_max=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_mean=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_max=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_min=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_mean=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_95th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99.9th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_25th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99.9th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_90th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_mean=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_95th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_min=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_75th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_max=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_max=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_90th_percentile=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_max=0 2023-10-15T16:14:17,043 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_98th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_25th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_median=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_median=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_75th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_mean=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99.9th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_median=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_90th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_mean=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_mean=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_75th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_num_ops=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99.9th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_median=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_75th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_median=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_median=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_98th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_95th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_75th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_95th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_num_ops=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_max=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_98th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_25th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_95th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99.9th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_num_ops=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_25th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99.9th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_median=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_90th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_25th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_median=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_min=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_max=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_90th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_max=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_75th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_25th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_98th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_min=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_mean=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,044 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99.9th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_25th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_min=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_mean=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_min=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_98th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_min=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_98th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99.9th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_median=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_num_ops=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_median=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_90th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_min=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_25th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99th_percentile=0 2023-10-15T16:14:17,045 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsCount=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_98th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_median=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestRatePerSecond=0.0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerMissCount=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_mean=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_median=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_98th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_95th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_median=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_90th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_5min_rate=0.0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_min=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_95th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitRatio=0.0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_75th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_90th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_num_ops=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_min=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_25th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestCount=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedOutputBytes=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_90th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkHitCount=1 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadOpsInCurThread=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_median=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestRatePerSecond=0.0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_25th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_min=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_num_ops=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_num_ops=26 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_90th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockBytesScannedCount=4389 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_max=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_75th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_max=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_mean=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_num_ops=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_25th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheEvictionCount=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99.9th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheCount=3 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:smallCompactionQueueLength=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCount=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:17,046 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_min=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_num_ops=15 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_mean=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowIncrementCount=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushQueueLength=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_mean=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_75th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:localBytesRead=108773 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushCount=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_median=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutateFailedCount=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_max=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorPoolAllocationBytes=1530880 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_98th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_min=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_median=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_min=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:updatesBlockedTime=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_median=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_mean=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitQueueLength=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_min=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_min=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_95th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCachingCount=3 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_25th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99.9th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileAge=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_median=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_75th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_max=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_95th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_num_ops=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_min=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cpRequestCount=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_90th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_median=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_max=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_num_ops=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALCount=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_98th_percentile=0 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitRatio=0.7272727272727273 2023-10-15T16:14:17,047 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataMissCount=1 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestCount=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissCount=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_75th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_mean=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheEvictionCount=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheFreeSize=922067336 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_95th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_median=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_75th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_max=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_median=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:activeScanners=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissCount=3 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_mean=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_max=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_max=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_95th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactionQueueLength=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_95th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_num_ops=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_max=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_1min_rate=0.0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_min=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_median=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_num_ops=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_median=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_mean=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_75th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_min=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99.9th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_median=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_25th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_25th_percentile=0 2023-10-15T16:14:17,048 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_90th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_num_ops=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheSize=679544 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_num_ops=6 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_num_ops=26 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_98th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_count=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_median=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowGetCount=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_median=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterRequestsCount=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_min=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_min=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_25th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_max=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileSize=414534 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_min=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileCount=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsCount=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99.9th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_98th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_num_ops=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedOutputBytes=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_max=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_num_ops=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_max=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticBloomSize=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexMissCount=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseInfoThresholdExceeded=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCount=3 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCount=3 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_max=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedToMob=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedMemstoreBytes=0 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheExpressHitPercent=72.72727272727273 2023-10-15T16:14:17,049 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionServerStartTime=1697385859285 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedInputBytes=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedFromMob=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_max=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRequestCount=47 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_25th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcGetRequestCount=3 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_90th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeCount=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCountHitPercent=72.72727272727273 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_max=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_mean=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_min=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_98th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_min=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsSize=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_mean=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowDeleteCount=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockedRequestCount=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_mean=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheAccessCount=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_min=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_median=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_min=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_90th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_min=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_95th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_90th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheEvictedCount=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_90th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_max=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:avgStoreFileAge=1819 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_25th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheCount=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitRequestCount=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_num_ops=1 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_min=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalBytesRead=108773 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedToMob=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_max=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedInputBytes=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheSize=679544 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,050 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileCount=473 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_mean=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcScanRequestCount=26 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreOffHeapSize=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_75th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheMissCount=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowPutCount=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_max=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreSize=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_75th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsSize=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_mean=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_min=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_75th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_median=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_98th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_median=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Context=regionserver 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:scannerLeaseExpiredCount=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:minStoreFileAge=1782 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitCount=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_98th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcFullScanRequestCount=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_mean=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitSuccessCount=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_mean=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_75th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_25th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCountPrimary=3 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_max=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataBlockCount=1 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_98th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_max=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_median=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsCount=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_mean=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsCount=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCachingCount=8 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFailedInsertionCount=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_90th_percentile=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsSize=0 2023-10-15T16:14:17,051 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean_rate=0.0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticIndexSize=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorUsedBufferCount=2 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_min=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99.9th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissRatio=0.2727272727272727 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_75th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_max=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_mean=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_95th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMultiRequestCount=1 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheSize=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexHitCount=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_max=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_98th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedOutputBytes=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocal=0.0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_98th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_max=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_num_ops=3 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99.9th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_25th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_min=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_mean=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_15min_rate=0.0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99.9th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_min=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_95th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_75th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_median=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_median=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_mean=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_95th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitCount=8 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:largeCompactionQueueLength=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_mean=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutatePassedCount=0 2023-10-15T16:14:17,052 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_max=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_95th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissRatio=0.0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsCount=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_min=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_mean=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_min=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:shortCircuitBytesRead=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99.9th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_median=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_min=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoMissCount=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileIndexSize=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99.9th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_mean=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerHitCount=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_98th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsSize=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCountPrimary=8 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_min=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_max=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99.9th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_90th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_90th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_num_ops=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_max=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_mean=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_max=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_min=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRowActionRequestCount=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMutateRequestCount=16 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheCount=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_25th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,053 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_mean=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataHitCount=7 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:zeroCopyBytesRead=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_98th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionCount=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_num_ops=1 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileCount=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_num_ops=1 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_median=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALSize=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_num_ops=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_mean=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_median=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheFreeSize=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:averageRegionSize=17474 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_95th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaMissCount=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_75th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:filteredReadRequestCount=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaHitCount=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_75th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_num_ops=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_95th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_median=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCount=8 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_num_ops=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.serverName=8c5d72df62de,45999,1697385859285 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReads=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowAppendCount=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:numReferenceFiles=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedFromMob=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSize=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_max=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_90th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_median=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_mean=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_98th_percentile=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadWins=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseWarnThresholdExceeded=0 2023-10-15T16:14:17,054 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_max=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_25th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoHitCount=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_median=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreHeapSize=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_25th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_min=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99.9th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_25th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_num_ops=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_90th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_max=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99.9th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_max=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_median=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsSize=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_median=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_median=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_min=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkMissCount=2 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_95th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFreeSize=922067336 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheHitPercent=0.0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemMaxM=2200.0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTerminated=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsWaiting=86 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.SessionId=null 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Context=jvm 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapUsedM=188.85072 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTotalExtraSleepTime=30 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Old Generation=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCount=11 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsBlocked=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogWarn=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsRunnable=83 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,055 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogInfo=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsNew=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapUsedM=325.47656 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTimedWaiting=119 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapMaxM=2200.0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Young Generation=11 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogError=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.ProcessName=NameNode 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogFatal=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillis=328 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemMaxM=2200.0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTerminated=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsWaiting=86 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.SessionId=null 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Context=jvm 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapUsedM=188.85118 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTotalExtraSleepTime=40 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Old Generation=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCount=11 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsBlocked=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogWarn=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsRunnable=83 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogInfo=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsNew=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapUsedM=325.47656 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTimedWaiting=119 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapMaxM=2200.0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Young Generation=11 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogError=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.ProcessName=DataNode 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogFatal=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillis=328 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_90th_percentile=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_25th_percentile=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_num_ops=14 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99.9th_percentile=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_90th_percentile=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_95th_percentile=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_median=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99th_percentile=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_max=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_95th_percentile=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_98th_percentile=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_98th_percentile=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_25th_percentile=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:fsChecksumFailureCount=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_median=0 2023-10-15T16:14:17,056 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_75th_percentile=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_min=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99th_percentile=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Context=regionserver 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_max=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_min=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_90th_percentile=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_75th_percentile=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_max=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99.9th_percentile=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_98th_percentile=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_mean=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_95th_percentile=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_num_ops=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_min=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_median=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99.9th_percentile=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_mean=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_num_ops=186 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_25th_percentile=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_mean=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_75th_percentile=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99th_percentile=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context=dfs 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.1=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.2=dfs 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.2=8c5d72df62de 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.1=dfs 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.1=8c5d72df62de 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.2=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Context=master 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:numMasterWALs=0 2023-10-15T16:14:17,057 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsCount=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_98th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_median=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestRatePerSecond=0.0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerMissCount=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_mean=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_median=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_98th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_95th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_median=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_90th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_5min_rate=0.0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_min=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_95th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitRatio=0.0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_75th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_90th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_num_ops=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_min=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_25th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestCount=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedOutputBytes=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_90th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkHitCount=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadOpsInCurThread=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_median=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestRatePerSecond=0.0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_25th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_min=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_num_ops=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_num_ops=2 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_90th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockBytesScannedCount=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_max=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_75th_percentile=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:17,058 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_max=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_mean=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_num_ops=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_25th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheEvictionCount=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99.9th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheCount=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:smallCompactionQueueLength=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCount=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_min=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_num_ops=6 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_mean=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowIncrementCount=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushQueueLength=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_mean=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_75th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:localBytesRead=108773 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushCount=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_median=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutateFailedCount=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_max=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorPoolAllocationBytes=199680 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_98th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_min=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_median=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_min=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:updatesBlockedTime=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_median=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_mean=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitQueueLength=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_min=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_min=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_95th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCachingCount=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_25th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99.9th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileAge=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_median=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_75th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_max=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_95th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_num_ops=1 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_min=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cpRequestCount=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_90th_percentile=0 2023-10-15T16:14:17,059 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_median=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_max=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_num_ops=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALCount=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_98th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitRatio=0.0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataMissCount=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestCount=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissCount=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_75th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_mean=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheEvictionCount=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheFreeSize=922070024 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_95th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_median=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_75th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_num_ops=1 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_max=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_median=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:activeScanners=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissCount=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_mean=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_max=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_max=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_95th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactionQueueLength=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_95th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_num_ops=1 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_max=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_1min_rate=0.0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_min=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_median=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_num_ops=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_median=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_mean=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_75th_percentile=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_min=0 2023-10-15T16:14:17,060 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99.9th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_median=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_25th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_25th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_90th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_num_ops=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheSize=676856 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_num_ops=2 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_98th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_count=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_median=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowGetCount=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_median=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterRequestsCount=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_min=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_min=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_25th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_max=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileSize=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_min=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileCount=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsCount=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99.9th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_98th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_num_ops=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedOutputBytes=5860 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_max=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_num_ops=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_max=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticBloomSize=0 2023-10-15T16:14:17,061 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexMissCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseInfoThresholdExceeded=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_max=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedToMob=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedMemstoreBytes=928 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheExpressHitPercent=0.0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionServerStartTime=1697385851561 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedInputBytes=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedFromMob=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_max=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRequestCount=11 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_25th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcGetRequestCount=3 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_90th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCountHitPercent=0.0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_max=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_mean=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_min=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_98th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_min=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsSize=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_mean=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowDeleteCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockedRequestCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_mean=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheAccessCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_min=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_median=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_min=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_90th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_min=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_95th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_90th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheEvictedCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_90th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_max=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:avgStoreFileAge=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_25th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitRequestCount=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_min=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalBytesRead=108773 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedToMob=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,062 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_max=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedInputBytes=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheSize=676856 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileCount=239 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_mean=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcScanRequestCount=2 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreOffHeapSize=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_75th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheMissCount=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowPutCount=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_max=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreSize=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_75th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsSize=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_mean=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_min=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_75th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_median=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_98th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_median=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Context=regionserver 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:scannerLeaseExpiredCount=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:minStoreFileAge=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitCount=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_98th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcFullScanRequestCount=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_mean=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitSuccessCount=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_mean=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_75th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_25th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCountPrimary=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_max=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataBlockCount=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_98th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_max=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_median=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsCount=0 2023-10-15T16:14:17,063 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_mean=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsCount=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCachingCount=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFailedInsertionCount=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_90th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsSize=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean_rate=0.0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticIndexSize=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_min=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99.9th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissRatio=0.0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_75th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_max=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_mean=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_95th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMultiRequestCount=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheSize=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexHitCount=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_max=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_98th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedOutputBytes=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocal=0.0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_98th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_max=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_num_ops=3 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99.9th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_25th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_min=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_mean=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_15min_rate=0.0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99.9th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_min=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_95th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_75th_percentile=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_median=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_median=0 2023-10-15T16:14:17,064 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_mean=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_95th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitCount=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:largeCompactionQueueLength=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_mean=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutatePassedCount=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_max=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_95th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissRatio=0.0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsCount=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_min=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_mean=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_min=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:shortCircuitBytesRead=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99.9th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_median=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_min=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoMissCount=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileIndexSize=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99.9th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_mean=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerHitCount=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_98th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsSize=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCountPrimary=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_min=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_max=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99.9th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_90th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_90th_percentile=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_num_ops=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,065 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_max=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_mean=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_max=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_min=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRowActionRequestCount=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMutateRequestCount=6 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheCount=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_25th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_mean=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataHitCount=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:zeroCopyBytesRead=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_98th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionCount=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_num_ops=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileCount=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_num_ops=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_median=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALSize=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_num_ops=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_mean=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_median=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheFreeSize=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:averageRegionSize=928 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_95th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaMissCount=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_75th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:filteredReadRequestCount=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaHitCount=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_75th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_num_ops=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_95th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_median=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCount=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_num_ops=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.serverName=8c5d72df62de,34443,1697385851561 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReads=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowAppendCount=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:numReferenceFiles=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedFromMob=0 2023-10-15T16:14:17,066 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSize=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_max=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_90th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_median=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_mean=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_98th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadWins=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseWarnThresholdExceeded=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_max=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_25th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoHitCount=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_median=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreHeapSize=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_25th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_min=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99.9th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_25th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_num_ops=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_90th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_max=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99.9th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_max=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_median=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsSize=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_median=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_median=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_min=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkMissCount=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_95th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFreeSize=922070024 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheHitPercent=0.0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:17,067 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsCount=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_98th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_median=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestRatePerSecond=0.0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerMissCount=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_mean=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_median=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_98th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_95th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_median=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_90th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_5min_rate=0.0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_min=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_95th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitRatio=0.0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_75th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_90th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_num_ops=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_min=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_25th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestCount=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedOutputBytes=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_90th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkHitCount=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadOpsInCurThread=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_median=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestRatePerSecond=0.0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_25th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_min=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_num_ops=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_num_ops=0 2023-10-15T16:14:17,068 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_90th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockBytesScannedCount=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_max=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_75th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_max=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_mean=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_num_ops=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_25th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheEvictionCount=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99.9th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheCount=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:smallCompactionQueueLength=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCount=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_min=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_num_ops=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_mean=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowIncrementCount=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushQueueLength=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_mean=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_75th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:localBytesRead=108773 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushCount=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_median=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutateFailedCount=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_max=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorPoolAllocationBytes=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_98th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_min=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_median=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_min=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:updatesBlockedTime=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_median=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_mean=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitQueueLength=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_min=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_min=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_95th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCachingCount=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_25th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99.9th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileAge=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_median=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_75th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_max=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_95th_percentile=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_num_ops=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_min=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cpRequestCount=0 2023-10-15T16:14:17,069 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_90th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_median=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_max=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_num_ops=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALCount=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_98th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitRatio=0.0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataMissCount=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestCount=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissCount=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_75th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_mean=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheEvictionCount=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheFreeSize=922070024 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_95th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_median=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_75th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_max=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_median=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:activeScanners=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissCount=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_mean=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_max=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_max=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_95th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactionQueueLength=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_95th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_num_ops=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_max=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_1min_rate=0.0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_min=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_median=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_num_ops=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_median=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_mean=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_75th_percentile=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_min=0 2023-10-15T16:14:17,070 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99.9th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_median=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_25th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_25th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_90th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_num_ops=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheSize=676856 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_num_ops=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_98th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_count=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_median=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowGetCount=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_median=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterRequestsCount=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_min=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_min=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_25th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_max=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileSize=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_min=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileCount=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsCount=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99.9th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_98th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_num_ops=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedOutputBytes=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_max=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_num_ops=0 2023-10-15T16:14:17,071 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_max=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticBloomSize=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexMissCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseInfoThresholdExceeded=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_max=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedToMob=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedMemstoreBytes=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheExpressHitPercent=0.0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionServerStartTime=1697385851811 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedInputBytes=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedFromMob=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_max=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRequestCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_25th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcGetRequestCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_90th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCountHitPercent=0.0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_max=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_mean=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_min=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_98th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_min=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsSize=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_mean=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowDeleteCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockedRequestCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_mean=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheAccessCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_min=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_median=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_min=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_90th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_min=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_95th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_90th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheEvictedCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_90th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_max=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:avgStoreFileAge=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_25th_percentile=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitRequestCount=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_min=0 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalBytesRead=108773 2023-10-15T16:14:17,072 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedToMob=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_max=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedInputBytes=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheSize=676856 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileCount=120 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_mean=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcScanRequestCount=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreOffHeapSize=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_75th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheMissCount=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowPutCount=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_max=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreSize=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_75th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsSize=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_mean=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_min=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_75th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_median=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_98th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_median=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Context=regionserver 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:scannerLeaseExpiredCount=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:minStoreFileAge=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitCount=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_98th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcFullScanRequestCount=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_mean=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitSuccessCount=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_mean=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_75th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_25th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCountPrimary=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_max=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataBlockCount=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_98th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_max=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_median=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsCount=0 2023-10-15T16:14:17,073 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_mean=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsCount=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCachingCount=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFailedInsertionCount=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_90th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsSize=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean_rate=0.0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticIndexSize=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorUsedBufferCount=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_min=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99.9th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissRatio=0.0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_75th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_max=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_mean=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_95th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMultiRequestCount=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheSize=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexHitCount=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_max=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_98th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedOutputBytes=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocal=0.0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_98th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_max=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_num_ops=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99.9th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_25th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_min=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_mean=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_15min_rate=0.0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99.9th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_min=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_95th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_75th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_median=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_median=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_mean=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_95th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitCount=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,074 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:largeCompactionQueueLength=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_mean=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutatePassedCount=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_max=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_95th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissRatio=0.0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsCount=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_min=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_mean=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_min=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:shortCircuitBytesRead=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99.9th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_median=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_min=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoMissCount=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileIndexSize=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99.9th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_mean=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerHitCount=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_98th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsSize=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCountPrimary=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_min=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_max=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99.9th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_90th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_90th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_num_ops=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_max=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_mean=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_max=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_min=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRowActionRequestCount=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMutateRequestCount=0 2023-10-15T16:14:17,075 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheCount=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_25th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_mean=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataHitCount=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:zeroCopyBytesRead=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_98th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionCount=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_num_ops=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileCount=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_num_ops=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_median=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALSize=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_num_ops=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_mean=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_median=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheFreeSize=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:averageRegionSize=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_95th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaMissCount=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_75th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:filteredReadRequestCount=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaHitCount=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_75th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_num_ops=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_95th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_median=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCount=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_num_ops=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.serverName=8c5d72df62de,40585,1697385851811 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReads=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowAppendCount=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:numReferenceFiles=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedFromMob=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSize=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_max=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_90th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_median=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_mean=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_98th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadWins=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseWarnThresholdExceeded=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,076 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_max=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_25th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoHitCount=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_median=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreHeapSize=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_25th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_min=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99.9th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_25th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_num_ops=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_90th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_max=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99.9th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_max=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_median=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsSize=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_median=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_median=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_min=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkMissCount=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_95th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFreeSize=922070024 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheHitPercent=0.0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:17,077 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsCount=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_98th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_median=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestRatePerSecond=0.0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerMissCount=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_mean=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_median=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_98th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_95th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_median=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_90th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_5min_rate=0.0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_min=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_95th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitRatio=0.0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_75th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_90th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_num_ops=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_min=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_25th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestCount=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedOutputBytes=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_90th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkHitCount=1 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadOpsInCurThread=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_median=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestRatePerSecond=0.0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_25th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_min=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_num_ops=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_num_ops=11 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_90th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockBytesScannedCount=6840 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_max=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_75th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_max=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_mean=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_num_ops=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_25th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheEvictionCount=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99.9th_percentile=0 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheCount=4 2023-10-15T16:14:17,078 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:smallCompactionQueueLength=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCount=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_min=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_num_ops=7 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_num_ops=1 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_mean=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowIncrementCount=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushQueueLength=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_mean=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_75th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:localBytesRead=108773 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushCount=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_median=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutateFailedCount=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_max=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorPoolAllocationBytes=599040 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_98th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_min=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_median=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_min=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:updatesBlockedTime=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_median=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_mean=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitQueueLength=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_min=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_min=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_95th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCachingCount=4 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_25th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99.9th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileAge=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_median=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_75th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_max=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_95th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_num_ops=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_min=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cpRequestCount=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_90th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_median=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_max=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_num_ops=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALCount=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_98th_percentile=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitRatio=0.72 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataMissCount=6 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestCount=0 2023-10-15T16:14:17,079 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissCount=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_75th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_mean=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheEvictionCount=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheFreeSize=922061632 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_95th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_median=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_75th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_max=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_median=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:activeScanners=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissCount=7 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_mean=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_max=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_max=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_95th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactionQueueLength=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_95th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_num_ops=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_max=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_1min_rate=0.0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_min=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_median=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_num_ops=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_median=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_mean=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_75th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_min=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99.9th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_median=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_25th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_25th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_90th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_num_ops=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheSize=685248 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_num_ops=7 2023-10-15T16:14:17,080 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_num_ops=11 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_98th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_count=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_median=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowGetCount=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_median=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterRequestsCount=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_min=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_min=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_25th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_num_ops=1 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_max=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileSize=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_min=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileCount=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsCount=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99.9th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_98th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_num_ops=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedOutputBytes=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_max=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_num_ops=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_max=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticBloomSize=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexMissCount=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseInfoThresholdExceeded=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCount=4 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCount=7 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_max=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedToMob=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedMemstoreBytes=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheExpressHitPercent=81.81818181818183 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionServerStartTime=1697385888915 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedInputBytes=18105 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedFromMob=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_max=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRequestCount=21 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_25th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcGetRequestCount=1 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_90th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeCount=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:17,081 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCountHitPercent=72.0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_max=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_mean=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_min=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_98th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_min=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsSize=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_mean=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowDeleteCount=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockedRequestCount=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_mean=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheAccessCount=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_min=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_median=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_min=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_90th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_min=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_95th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_90th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheEvictedCount=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_90th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_max=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:avgStoreFileAge=12090 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_25th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheCount=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitRequestCount=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_num_ops=1 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_min=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalBytesRead=108773 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedToMob=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_max=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedInputBytes=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheSize=685248 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileCount=339 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_mean=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcScanRequestCount=11 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreOffHeapSize=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_75th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:17,082 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheMissCount=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowPutCount=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_max=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreSize=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_75th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsSize=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_mean=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_min=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_75th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_median=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_98th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_median=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Context=regionserver 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:scannerLeaseExpiredCount=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:minStoreFileAge=2959 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitCount=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_98th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcFullScanRequestCount=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_mean=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitSuccessCount=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_mean=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_75th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_25th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCountPrimary=7 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_max=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataBlockCount=3 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_98th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_max=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_median=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsCount=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_mean=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsCount=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCachingCount=18 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFailedInsertionCount=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_90th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsSize=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean_rate=0.0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticIndexSize=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_min=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99.9th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissRatio=0.28 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_75th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_max=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_mean=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,083 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_95th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMultiRequestCount=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheSize=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexHitCount=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_max=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_98th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedOutputBytes=5921 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocal=0.0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_98th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_max=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_num_ops=1 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99.9th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_25th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_min=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_mean=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_15min_rate=0.0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99.9th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_min=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_95th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_75th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_median=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_median=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_mean=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_95th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitCount=18 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:largeCompactionQueueLength=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_mean=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutatePassedCount=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_max=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_95th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissRatio=0.0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsCount=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_min=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_mean=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_min=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:shortCircuitBytesRead=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99.9th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99th_percentile=0 2023-10-15T16:14:17,084 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_median=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_min=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoMissCount=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileIndexSize=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99.9th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_mean=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerHitCount=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_98th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsSize=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCountPrimary=18 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_min=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_max=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99.9th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_90th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_90th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_num_ops=1 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_max=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_mean=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_max=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_min=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRowActionRequestCount=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMutateRequestCount=8 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheCount=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_25th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationBytes=89913 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_mean=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataHitCount=17 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:zeroCopyBytesRead=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_98th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionCount=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_num_ops=1 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileCount=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_num_ops=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_median=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALSize=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_num_ops=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_mean=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_median=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheFreeSize=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:averageRegionSize=32082 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_95th_percentile=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaMissCount=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:17,085 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_75th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:filteredReadRequestCount=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaHitCount=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_75th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_num_ops=1 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_95th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_median=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCount=18 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_num_ops=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.serverName=8c5d72df62de,35419,1697385888915 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReads=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowAppendCount=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:numReferenceFiles=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedFromMob=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSize=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_max=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_90th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_median=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_mean=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_98th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadWins=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseWarnThresholdExceeded=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_max=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_25th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoHitCount=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_median=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreHeapSize=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_25th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_min=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99.9th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_25th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_num_ops=1 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_90th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_max=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99.9th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_max=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_median=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsSize=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_median=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_median=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_min=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkMissCount=1 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,086 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_95th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFreeSize=922061632 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheHitPercent=0.0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_98th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_num_ops=2 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.walReaderEditsBufferUsage=80117 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_25th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfHFileRefsQueue=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logReadInBytes=83847 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_75th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfLogQueue=-3 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_90th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99.9th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_num_ops=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedOps=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_min=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedHFiles=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_min=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_mean=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBytes=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_90th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedOps=1000 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedBatches=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_98th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_max=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_95th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedBatches=1 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_mean=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_95th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedHFiles=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_median=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.uncleanlyClosedLogs=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.closedLogsWithUnknownFileLength=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_max=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.repeatedLogFileBytes=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedRecoverQueues=5 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Context=regionserver 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedRecoverQueues=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.restartedLogReading=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.ignoredUncleanlyClosedLogContentsInBytes=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsRead=40 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.numInitializing=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99.9th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_75th_percentile=0 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsFiltered=38 2023-10-15T16:14:17,087 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_25th_percentile=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedLogs=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.failedBatches=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBatches=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_median=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWrite=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosNumOps=26 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRemoved=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsNumOps=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksCached=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsAvgTime=94.0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalNumOps=9 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesRead=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpAvgTime=6326.928571428572 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesAvgTime=0.0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionTasks=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncCount=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:NumProcessedCommands=10 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpNumOps=27 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromLocalClient=28 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:VolumeFailures=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosNumOps=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesRead=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosAvgTime=561936.0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalReadTime=9 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesRead=52810 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsAvgTime=1.7142857142857142 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromRemoteClient=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksUncached=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DatanodeNetworkErrors=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromLocalClient=27 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksInPendingIBR=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockVerificationFailures=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosNumOps=54 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesWrite=0 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosNumOps=54 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.SessionId=null 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsNumOps=1 2023-10-15T16:14:17,088 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesWritten=620489 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRead=27 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReplicated=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Context=dfs 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksVerified=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromRemoteClient=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosAvgTime=20303.65 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsNumOps=9 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksWritten=28 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsAvgTime=1.0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalWriteTime=12 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcDecodingTimeNanos=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesNumOps=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosNumOps=1285 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpNumOps=28 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosAvgTime=23635.714285714286 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesWritten=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosAvgTime=10588.8 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpNumOps=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumFailedVolumes=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheCapacity=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheUsed=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:FailedStorageLocations=[Ljava.lang.String;@174e9d70 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksCached=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:LastVolumeFailureDate=0 2023-10-15T16:14:17,089 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Capacity=1054589059072 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToUncache=7 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:DfsUsed=625022 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Remaining=941197720566 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToCache=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numRegionSizeReports=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_min=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_num_ops=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_98th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_num_ops=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numNamespaceInQuotaViolation=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_75th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_95th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_90th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_median=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_max=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.namespaceSpaceQuotaOverview=[] 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numTablesInQuotaViolation=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_num_ops=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99.9th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_max=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_98th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_median=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_min=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_max=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Context=master 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99.9th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_95th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_median=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_mean=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_mean=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_max=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_25th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_mean=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_min=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_75th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_mean=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_90th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_min=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_median=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.tableSpaceQuotaOverview=[] 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numSpaceQuotas=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_25th_percentile=0 2023-10-15T16:14:17,090 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_num_ops=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenSubmittedCount=14 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignSubmittedCount=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_num_ops=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99.9th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_median=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_max=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_95th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_95th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_min=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_mean=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_min=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_median=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_min=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveSubmittedCount=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_25th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_mean=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99.9th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_max=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnFS=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerRegions=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_75th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_95th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignSubmittedCount=13 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_75th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_98th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_num_ops=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_90th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignFailedCount=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_num_ops=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_90th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignFailedCount=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_25th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_98th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_98th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveFailedCount=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_mean=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_75th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeFailedCount=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Context=master 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_num_ops=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseSubmittedCount=1 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:holes=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_75th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99.9th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_95th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_95th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeSubmittedCount=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99.9th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_median=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99.9th_percentile=0 2023-10-15T16:14:17,091 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_mean=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCount=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenSubmittedCount=1 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseFailedCount=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_max=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:inconsistentRegions=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_90th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenFailedCount=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnRS=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99.9th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_75th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_75th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_min=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_25th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerOpenRegions=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_95th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_25th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:overlaps=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenFailedCount=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_90th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_98th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitFailedCount=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_max=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_98th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_mean=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_num_ops=13 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_median=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_mean=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCountOverThreshold=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_min=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_75th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_90th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_num_ops=1 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_75th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_95th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_25th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_median=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_25th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitSubmittedCount=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_mean=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritOldestAge=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:emptyRegionInfoRegions=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_max=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_25th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99.9th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_90th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_min=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99.9th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_num_ops=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_max=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_95th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_median=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_max=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_90th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_25th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_25th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_min=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_90th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_95th_percentile=0 2023-10-15T16:14:17,092 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_max=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99th_percentile=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_num_ops=1 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_min=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:operationCount=15 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_98th_percentile=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:deadServerOpenRegions=3 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_mean=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99th_percentile=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_median=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_min=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99.9th_percentile=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_98th_percentile=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_median=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_max=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_mean=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_num_ops=14 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_98th_percentile=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_90th_percentile=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_median=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_98th_percentile=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_75th_percentile=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumFailedVolumes=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheUsed=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheCapacity=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksCached=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:LastVolumeFailureDate=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Capacity=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Context=FSDatasetState 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data2]'} 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:DfsUsed=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Remaining=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToCache=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToUnCache=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalDataFileIos=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalFileIoErrors=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateNumOps=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateNumOps=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateNumOps=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateNumOps=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateNumOps=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateNumOps=0 2023-10-15T16:14:17,093 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Context=dfs 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalMetadataOperations=0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsNumOps=17 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportNumOps=14 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsNumOps=54 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoAvgTime=0.6666666666666666 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoNumOps=675 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestNumOps=3 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedNumOps=159 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateNumOps=60 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.port=39027 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeNumOps=3 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportAvgTime=0.3333333333333333 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Context=rpcdetailed 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameNumOps=41 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportAvgTime=0.21428571428571427 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateAvgTime=5.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteNumOps=60 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingNumOps=206 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsNumOps=322 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportNumOps=3 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeAvgTime=0.6666666666666666 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionNumOps=1 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockNumOps=49 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteNumOps=33 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateNumOps=1 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesNumOps=16 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeNumOps=1 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatNumOps=600 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseNumOps=19 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseNumOps=5 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatAvgTime=0.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemMaxM=2200.0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTerminated=0 2023-10-15T16:14:17,094 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsWaiting=86 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.SessionId=null 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Context=jvm 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapUsedM=188.86484 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTotalExtraSleepTime=55 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Old Generation=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCount=11 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsBlocked=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogWarn=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsRunnable=83 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogInfo=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsNew=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapUsedM=327.47656 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTimedWaiting=119 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapMaxM=2200.0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Young Generation=11 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogError=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.ProcessName=DataNode 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogFatal=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillis=328 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDataNodeMessageCount=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastWrittenTransactionId=431 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:SnapshottableDirectories=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEncryptionZones=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReconstructionBlocks=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptBlocks=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastCheckpointTime=1697385856136 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ReadLockLongHoldCount=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumActiveClients=1 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastLogRoll=431 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumTimedOutPendingReconstructions=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:VolumeFailuresTotal=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ProvidedCapacityTotal=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingECBlockGroups=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedGB=0.0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecommissioningDataNodes=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.HAState=active 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionBlocks=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyECBlockGroups=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:FilesTotal=109 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ScheduledReplicationBlocks=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureECBlockGroups=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptECBlockGroups=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MillisSinceLastLoadedEdits=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalECBlockGroups=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExcessBlocks=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalLoad=9 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,095 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExpiredHeartbeats=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInServiceLiveDataNodes=3 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumStaleStorages=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:WriteLockLongHoldCount=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlocksTotal=42 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:UnderReplicatedBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicatedBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsed=1391792 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalReplicatedBlocks=42 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:StaleDataNodes=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumFilesUnderConstruction=1 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemaining=2823754896354 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PostponedMisreplicatedBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedNonDFS=178447369040 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LockQueueLength=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptReplicatedBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Context=dfs 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomDeadDataNodes=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:Snapshots=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionECBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.TotalSyncTimes=11 5 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReplicationBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotal=3163767177216 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDeadDataNodes=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplOneBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalSyncCount=388 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastCheckpoint=431 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicationOneBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumLiveDataNodes=3 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlockCapacity=4194304 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotalGB=2946.0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyBlocks=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomLiveDataNodes=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemainingGB=2630.0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:ReceivedBytes=15456 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcSlowCalls=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationSuccesses=46 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumDroppedConnections=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcClientBackoff=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationFailures=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumOpenConnections=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.port=35085 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:SentBytes=9822 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeNumOps=46 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationSuccesses=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:CallQueueLength=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Context=rpc 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationFailures=0 2023-10-15T16:14:17,096 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeNumOps=46 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeNumOps=46 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:receivedBytes=288613 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationSuccesses=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_max=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFailures=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numOpenConnections=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_num_ops=1195 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_median=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationSuccesses=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_num_ops=1195 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_num_ops=1195 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_max=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_max=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_median=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.otherExceptions=5 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Context=regionserver 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_median=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_min=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_median=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_max=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_min=0 2023-10-15T16:14:17,097 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_min=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveHandler=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_num_ops=1195 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_mean=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReadHandler=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_median=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_num_ops=1195 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_mean=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callDropped=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions=5 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFallbacks=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationFailures=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_min=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_max=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:queueSize=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveScanHandler=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_min=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:sentBytes=44933 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_mean=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_75th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_max=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_90th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_min=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_median=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_num_ops=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOnHeapSize=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_num_ops=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_num_ops=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_median=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_num_ops=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_75th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_min=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOffHeapSize=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_25th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_75th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_mean=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_25th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_25th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_mean=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tunerDoNothingCounter=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_25th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_num_ops=0 2023-10-15T16:14:17,098 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:aboveHeapOccupancyLowWaterMarkCounter=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_98th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreSize=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_min=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_max=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_90th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_mean=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_75th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Context=regionserver 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99.9th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_mean=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_min=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockCacheSize=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_90th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockedFlushGauge=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_max=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_median=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_98th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:unblockedFlushGauge=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_max=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_max=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_num_ops=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_min=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_95th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_90th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_95th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_max=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_median=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_98th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_min=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99.9th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_median=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_median=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_95th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_mean=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_95th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_98th_percentile=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalDataFileIos=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalFileIoErrors=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateNumOps=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateNumOps=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateNumOps=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateNumOps=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateNumOps=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateNumOps=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Context=dfs 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalMetadataOperations=0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,099 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:HttpPort=null 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XceiverCount=3 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"2","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"599"}] 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:ClusterId=testClusterID 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3":{"numBlocks":21,"usedSpace":108926,"freeSpace":470665322496,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4":{"numBlocks":21,"usedSpace":516096,"freeSpace":470531349494,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":133973002}} 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DatanodeHostname=127.0.0.1 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SecurityEnabled=false 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:RpcPort=0 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DataPort=0 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SoftwareVersion=3.2.4 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SlowDisks=null 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XmitsInProgress=0 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DNStartedTimeInMillis=1697385857021 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsBatchedInSync=44 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsNumOps=387 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsAvgTime=0.11764705882352941 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportNumOps=0 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeNumOps=0 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteSnapshotOps=0 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsAvgTime=0.09523809523809523 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AllowSnapshotOps=0 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:RenameSnapshotOps=0 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeAvgTime=0.08333333333333333 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesCreated=168 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesAppended=0 2023-10-15T16:14:17,100 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportAvgTime=0.0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageAvgTime=0.0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeNumOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeNumOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeNumOps=126 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeAvgTime=0.0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumTimesReReplicationNotScheduled=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SafeModeTime=1 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalNumOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetListingOps=132 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.ProcessName=NameNode 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageNumOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TotalFileOps=1303 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSymlinkOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AddBlockOps=49 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DisallowSnapshotOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteFileOps=23 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsNumOps=431 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSnapshotOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PendingEditsCount=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ListSnapshottableDirOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockReceivedAndDeletedOps=159 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.SessionId=null 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditNumOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FsImageLoadTime=23 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateFileOps=60 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Context=dfs 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesRenamed=41 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TimeoutReReplications=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsQueued=1 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedNumOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageNumOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageAvgTime=0.0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesInGetListingOps=146 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FileInfoOps=675 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetLinkTargetOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetAdditionalDatanodeOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsBatched=68 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesDeleted=39 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SuccessfulReReplications=21 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetBlockLocations=322 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesTruncated=1 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SnapshotDiffReportOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportNumOps=6 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeNumOps=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditAvgTime=0.0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportAvgTime=0.3333333333333333 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumFailedVolumes=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheUsed=0 2023-10-15T16:14:17,101 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheCapacity=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksCached=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:LastVolumeFailureDate=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Capacity=1054589059072 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Context=FSDatasetState 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:DfsUsed=383910 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Remaining=941226220534 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToCache=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToUnCache=7 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalDataFileIos=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalFileIoErrors=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateNumOps=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateNumOps=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateNumOps=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateNumOps=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateNumOps=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateNumOps=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Context=dfs 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalMetadataOperations=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessNumOps=1 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureNumOps=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailures=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsNumOps=2 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailuresTotal=0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Context=ugi 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsAvgTime=6.0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessAvgTime=10.0 2023-10-15T16:14:17,102 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureAvgTime=0.0 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfSnapshottableDirs=0 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheCapacity=0 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheUsed=0 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ProvidedCapacity=0 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DistinctVersionCount=1 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocksWithReplicationFactorOne=0 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentRemaining=89.25123 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentUsed=4.3991608E-5 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:TotalBlocks=42 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NonDfsUsedSpace=178501124944 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Safemode= 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:UpgradeFinalized=true 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:EnteringMaintenanceNodes={} 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentBlockPoolUsed=4.3991608E-5 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:LiveNodes={"127.0.0.1:40557":{"infoAddr":"127.0.0.1:35357","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:40557","lastContact":2,"usedSpace":625022,"adminState":"In Service","nonDfsUsedSpace":59479094914,"capacity":1054589059072,"numBlocks":41,"version":"3.2.4","used":625022,"remaining":941254826998,"blockScheduled":1,"blockPoolUsed":625022,"blockPoolUsedPercent":5.9266877E-5,"volfails":0,"lastBlockReport":9},"127.0.0.1:46365":{"infoAddr":"127.0.0.1:35919","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46365","lastContact":0,"usedSpace":383910,"adminState":"In Service","nonDfsUsedSpace":59531428954,"capacity":1054589059072,"numBlocks":41,"version":"3.2.4","used":383910,"remaining":941202734070,"blockScheduled":1,"blockPoolUsed":383910,"blockPoolUsedPercent":3.6403755E-5,"volfails":0,"lastBlockReport":10},"127.0.0.1:46399":{"infoAddr":"127.0.0.1:44887","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46399","lastContact":1,"usedSpace":382860,"adminState":"In Service","nonDfsUsedSpace":59490601076,"capacity":1054589059072,"numBlocks":41,"version":"3.2.4","used":382860,"remaining":941243575286,"blockScheduled":1,"blockPoolUsed":382860,"blockPoolUsedPercent":3.6304187E-5,"volfails":0,"lastBlockReport":9}} 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CompileInfo=2022-07-12T11:58Z by ubuntu from branch-3.2.4 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolId=BP-1531809096-172.17.0.3-1697385856100 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ClusterId=testClusterID 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DeadNodes={} 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:JournalTransactionInfo={"MostRecentCheckpointTxId":"0","LastAppliedOrWrittenTxId":"431"} 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Free=2823701136354 2023-10-15T16:14:17,104 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolUsedSpace=1391792 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Threads=288 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocks=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirStatuses={"active":{"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":"IMAGE_AND_EDITS","/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":"IMAGE_AND_EDITS"},"failed":{}} 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:SoftwareVersion=3.2.4 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:VerifyECWithTopologyResult={"isSupported":"false","resultMessage":"The number of DataNodes (3) is less than the minimum required number of DataNodes (9) for the erasure coding policies: RS-6-3-1024k"} 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Used=1391792 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Total=3163767177216 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NodeUsage={"nodeUsage":{"min":"0.00%","median":"0.00%","max":"0.00%","stdDev":"0.00%"}} 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NNStartedTimeInMillis=1697385856352 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DecomNodes={} 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CorruptFiles=[] 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirSize={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":737,"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":737} 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameJournalStatus=[{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"},{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"}] 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemMaxM=2200.0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTerminated=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsWaiting=86 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.SessionId=null 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Context=jvm 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapUsedM=188.87222 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTotalExtraSleepTime=36 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Old Generation=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCount=11 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsBlocked=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogWarn=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsRunnable=83 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogInfo=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsNew=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapUsedM=328.47656 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTimedWaiting=119 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapMaxM=2200.0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Young Generation=11 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogError=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.ProcessName=DataNode 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogFatal=0 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillis=328 2023-10-15T16:14:17,105 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlocksTotal=42 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TopUserOpCounts={"timestamp":"2023-10-15T16:14:17+0000","windows":[{"ops":[{"opType":"*","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"getfileinfo","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]}],"windowLenMs":60000},{"ops":[{"opType":"*","totalCount":4,"allUsers":[{"user":"jenkins","count":4}],"topUsers":[{"user":"jenkins","count":4}]},{"opType":"listStatus","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"getfileinfo","totalCount":3,"allUsers":[{"user":"jenkins","count":3}],"topUsers":[{"user":"jenkins","count":3}]}],"windowLenMs":300000},{"ops":[{"opType":"*","totalCount":1424,"allUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":300},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":300},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}]},{"opType":"datanodeReport","totalCount":14,"allUsers":[{"user":"jenkins","count":14}],"topUsers":[{"user":"jenkins","count":14}]},{"opType":"listStatus","totalCount":206,"allUsers":[{"user":"jenkins","count":66},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}],"topUsers":[{"user":"jenkins","count":66},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}]},{"opType":"safemode_get","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"delete","totalCount":33,"allUsers":[{"user":"jenkins.hfs.4","count":13},{"user":"jenkins","count":12},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}],"topUsers":[{"user":"jenkins.hfs.4","count":13},{"user":"jenkins","count":12},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}]},{"opType":"getfileinfo","totalCount":675,"allUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":136},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}],"topUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":136},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}]},{"opType":"truncate","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"rename","totalCount":41,"allUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":12},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":12},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}]},{"opType":"mkdirs","totalCount":54,"allUsers":[{"user":"jenkins","count":32},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}],"topUsers":[{"user":"jenkins","count":32},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}]},{"opType":"create","totalCount":60,"allUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":17},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":17},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}]},{"opType":"setPermission","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"setTimes","totalCount":16,"allUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}]},{"opType":"open","totalCount":322,"allUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":4}],"topUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":4}]}],"windowLenMs":1500000}]} 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:SnapshotStats={"SnapshottableDirectories":0,"Snapshots":0} 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:UnderReplicatedBlocks=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityUsed=1391792 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEncryptionZones=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityRemaining=2823701136354 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReconstructionBlocks=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomDeadDataNodes=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FSState=Operational 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:VolumeFailuresTotal=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ProvidedCapacityTotal=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecommissioningDataNodes=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReplicationBlocks=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncTimes=11 5 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingDeletionBlocks=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityTotal=3163767177216 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FilesTotal=109 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ScheduledReplicationBlocks=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDeadDataNodes=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncCount=388 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FsLockQueueLength=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:MaxObjects=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalLoad=9 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleDataNodes=0 2023-10-15T16:14:17,107 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumLiveDataNodes=3 2023-10-15T16:14:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlockDeletionStartTime=1697385856352 2023-10-15T16:14:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInServiceLiveDataNodes=3 2023-10-15T16:14:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:LowRedundancyBlocks=0 2023-10-15T16:14:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomLiveDataNodes=0 2023-10-15T16:14:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleStorages=0 2023-10-15T16:14:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(109): << End DataNode metrics dump 2023-10-15T16:14:17,182 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:17,298 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(83): >> Begin DataNode metrics dump 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_num_ops=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_mean=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_num_ops=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_25th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_95th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_90th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_90th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_max=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_98th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_25th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_75th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_min=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_75th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_min=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_max=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_max=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_25th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99.9th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_75th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_mean=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99.9th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_75th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_median=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_95th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_98th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99.9th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_median=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_25th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Context=master 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_median=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_95th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_98th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_90th_percentile=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_mean=0 2023-10-15T16:14:17,299 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99.9th_percentile=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_95th_percentile=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_min=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_max=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99th_percentile=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_98th_percentile=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_mean=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_num_ops=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_min=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_median=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_num_ops=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_90th_percentile=0 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:HttpPort=null 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XceiverCount=3 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:17,300 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"0","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"600"}] 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:ClusterId=testClusterID 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1":{"numBlocks":21,"usedSpace":266240,"freeSpace":470661980160,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2":{"numBlocks":21,"usedSpace":117670,"freeSpace":470528007158,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":133973002}} 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DatanodeHostname=127.0.0.1 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SecurityEnabled=false 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:RpcPort=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DataPort=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SoftwareVersion=3.2.4 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SlowDisks=null 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XmitsInProgress=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DNStartedTimeInMillis=1697385856691 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Context=regionserver 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.port=44717 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Context=rpcdetailed 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthNumOps=50 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpNumOps=4 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWrite=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosNumOps=44 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRemoved=7 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalAvgTime=0.9795918367346939 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsNumOps=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksCached=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsAvgTime=2.0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DataNodeActiveXceiversCount=1 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalNumOps=200 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsNumOps=53 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesRead=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpAvgTime=3792.75 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesAvgTime=0.0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionTasks=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncCount=0 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:NumProcessedCommands=201 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpNumOps=92 2023-10-15T16:14:17,301 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromLocalClient=48 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:VolumeFailures=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosNumOps=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesRead=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosAvgTime=1433727.0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalReadTime=16 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesRead=298317 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsAvgTime=1.0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromRemoteClient=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksUncached=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DatanodeNetworkErrors=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromLocalClient=92 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksInPendingIBR=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockVerificationFailures=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosNumOps=184 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesWrite=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosNumOps=184 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.SessionId=null 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsNumOps=1 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesWritten=973747 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRead=92 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReplicated=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Context=dfs 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksVerified=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromRemoteClient=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosAvgTime=20216.833333333332 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsNumOps=200 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksWritten=48 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsAvgTime=0.9183673469387755 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalWriteTime=21 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcDecodingTimeNanos=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesNumOps=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosNumOps=1700 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpNumOps=48 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosAvgTime=5276.454545454545 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesWritten=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosAvgTime=12045.5 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpNumOps=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,302 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumFailedVolumes=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheUsed=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheCapacity=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksCached=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:LastVolumeFailureDate=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Capacity=1054589059072 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Context=FSDatasetState 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:DfsUsed=625022 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Remaining=941225311222 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToCache=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToUnCache=7 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheCleared=1 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheHit=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Context=rpc 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheUpdated=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalDataFileIos=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalFileIoErrors=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateNumOps=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateNumOps=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateNumOps=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateNumOps=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateNumOps=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateNumOps=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Context=dfs 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalMetadataOperations=0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,303 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:MissingECBlockGroups=0 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:LowRedundancyECBlockGroups=0 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:BytesInFutureECBlockGroups=0 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:CorruptECBlockGroups=0 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:PendingDeletionECBlocks=0 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:TotalECBlockGroups=0 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheCleared=1 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheHit=0 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Context=rpc 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheUpdated=0 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:LastHATransitionTime=0 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:HostAndPort=localhost:39027 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:State=active 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowPeersReport=null 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowDisksReport=null 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:NNRole=NameNode 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:BytesWithFutureGenerationStamps=0 2023-10-15T16:14:17,304 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SecurityEnabled=false 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.port=35753 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Context=rpcdetailed 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthNumOps=55 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:ReceivedBytes=18480 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcSlowCalls=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationSuccesses=55 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumDroppedConnections=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcClientBackoff=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationFailures=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumOpenConnections=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.port=35753 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:SentBytes=11751 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeNumOps=55 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationSuccesses=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:CallQueueLength=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Context=rpc 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationFailures=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeNumOps=55 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeNumOps=55 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:ReceivedBytes=16800 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcSlowCalls=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationSuccesses=50 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumDroppedConnections=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcClientBackoff=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationFailures=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumOpenConnections=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.port=44717 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:SentBytes=10673 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeNumOps=50 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationSuccesses=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:CallQueueLength=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Context=rpc 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationFailures=0 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeNumOps=50 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeNumOps=50 2023-10-15T16:14:17,305 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemMaxM=2200.0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTerminated=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsWaiting=86 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.SessionId=null 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Context=jvm 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapUsedM=188.82245 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTotalExtraSleepTime=51 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Old Generation=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCount=11 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsBlocked=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogWarn=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsRunnable=83 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogInfo=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsNew=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapUsedM=323.47656 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTimedWaiting=119 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapMaxM=2200.0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Young Generation=11 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogError=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.ProcessName=NameNode 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogFatal=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillis=328 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumFailedVolumes=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheUsed=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheCapacity=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksCached=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:LastVolumeFailureDate=0 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Capacity=1054589059072 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Context=FSDatasetState 2023-10-15T16:14:17,306 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:DfsUsed=382860 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Remaining=941225311222 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToCache=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToUnCache=7 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemMaxM=2200.0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTerminated=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsWaiting=86 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.SessionId=null 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Context=jvm 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapUsedM=188.82245 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTotalExtraSleepTime=22 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Old Generation=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCount=11 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsBlocked=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogWarn=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsRunnable=84 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogInfo=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsNew=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapUsedM=323.47656 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTimedWaiting=118 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapMaxM=2200.0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Young Generation=11 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogError=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.ProcessName=DataNode 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogFatal=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillis=328 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_num_ops=1891 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99th_percentile=1 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_max=4 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_mean=206 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:appendCount=1891 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:lowReplicaRollRequest=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99th_percentile=332 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99.9th_percentile=4 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:sizeRollRequest=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendSize=297021 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_max=1 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowAppendCount=0 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:successfulLogRolls=24 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99.9th_percentile=333 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_TimeRangeCount_0-1=11 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_25th_percentile=36 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendSize=9557 2023-10-15T16:14:17,307 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_90th_percentile=1 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendCount=1011 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_25th_percentile=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_10-100=4 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99th_percentile=4 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_98th_percentile=4 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_75th_percentile=298 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_75th_percentile=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_max=334 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_95th_percentile=4 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_mean=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendCount=48 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_100-1000=7 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:errorRollRequest=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:rollRequest=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_num_ops=1804 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_median=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_3-10=1 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_98th_percentile=1 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_min=36 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_min=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_median=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Context=regionserver 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_0-1=10 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendCount=45 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_95th_percentile=326 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendSize=74104 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_median=262 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_98th_percentile=331 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_25th_percentile=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_75th_percentile=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_mean=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99.9th_percentile=1 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendSize=14494 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:writtenBytes=395176 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_90th_percentile=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_95th_percentile=1 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowSyncRollRequest=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_num_ops=1891 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_90th_percentile=319 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_min=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendCount=787 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicationOneBlocks=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:CorruptReplicatedBlocks=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicatedBlocks=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:TotalReplicatedBlocks=42 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalDataFileIos=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,308 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalFileIoErrors=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Context=dfs 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalMetadataOperations=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotAvgTime=0.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Context=metricssystem 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSources=67 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:DroppedPubAll=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishAvgTime=0.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSources=67 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSinks=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSinks=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheCleared=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheHit=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Context=rpc 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheUpdated=134 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWrite=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosNumOps=28 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRemoved=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsNumOps=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksCached=0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsAvgTime=3.0 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:17,309 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalNumOps=9 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesRead=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpAvgTime=9.0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpAvgTime=6810.7692307692305 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesAvgTime=0.0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionTasks=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncCount=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:NumProcessedCommands=10 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpNumOps=31 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromLocalClient=28 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:VolumeFailures=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosNumOps=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesRead=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosAvgTime=554484.75 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalReadTime=7 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesRead=57958 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsAvgTime=1.5714285714285714 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromRemoteClient=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksUncached=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DatanodeNetworkErrors=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromLocalClient=31 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksInPendingIBR=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockVerificationFailures=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosNumOps=62 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesWrite=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosNumOps=62 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.SessionId=null 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsNumOps=1 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesWritten=620489 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRead=31 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReplicated=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Context=dfs 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksVerified=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromRemoteClient=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosAvgTime=19658.18181818182 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsNumOps=9 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksWritten=28 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsAvgTime=1.0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalWriteTime=12 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcDecodingTimeNanos=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesNumOps=0 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosNumOps=1285 2023-10-15T16:14:17,310 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpNumOps=28 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosAvgTime=4301.205882352941 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesWritten=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosAvgTime=9971.454545454546 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpAvgTime=0.8181818181818182 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpNumOps=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemMaxM=2200.0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTerminated=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsWaiting=86 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.SessionId=null 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Context=jvm 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapUsedM=188.83234 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTotalExtraSleepTime=62 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Old Generation=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCount=11 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsBlocked=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogWarn=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsRunnable=83 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogInfo=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsNew=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapUsedM=323.47656 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTimedWaiting=119 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapMaxM=2200.0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Young Generation=11 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogError=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.ProcessName=DataNode 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogFatal=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillis=328 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Context=regionserver 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:numTables=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99.9th_percentile=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_90th_percentile=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_95th_percentile=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterRequests=1101 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99th_percentile=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterWriteRequests=1059 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_mean=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterReadRequests=74 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_min=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_25th_percentile=0 2023-10-15T16:14:17,311 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashSubmittedCount=4 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashFailedCount=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_98th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Context=master 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_num_ops=2 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_75th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_max=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_median=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:receivedBytes=6048147 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationSuccesses=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_max=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFailures=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numOpenConnections=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_num_ops=1266 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_median=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationSuccesses=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_num_ops=1266 2023-10-15T16:14:17,314 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_num_ops=1266 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_max=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_max=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_median=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.otherExceptions=3 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Context=master 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_median=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_min=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_median=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_max=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_min=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_min=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveHandler=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_num_ops=1266 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_mean=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReadHandler=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_median=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_num_ops=1266 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_mean=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callDropped=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions=3 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFallbacks=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationFailures=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_min=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:14:17,315 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_max=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:queueSize=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveScanHandler=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_min=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:sentBytes=16055 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumFailedVolumes=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheUsed=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheCapacity=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksCached=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:LastVolumeFailureDate=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Capacity=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Context=FSDatasetState 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data4]'} 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:DfsUsed=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Remaining=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToCache=0 2023-10-15T16:14:17,316 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToUnCache=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:HttpPort=null 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XceiverCount=3 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"1","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"598"}] 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:ClusterId=testClusterID 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5":{"numBlocks":21,"usedSpace":266240,"freeSpace":470661709824,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6":{"numBlocks":21,"usedSpace":116620,"freeSpace":470527736822,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":133973002}} 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DatanodeHostname=127.0.0.1 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SecurityEnabled=false 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:RpcPort=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DataPort=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SoftwareVersion=3.2.4 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SlowDisks=null 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XmitsInProgress=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DNStartedTimeInMillis=1697385857417 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalDataFileIos=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalFileIoErrors=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateNumOps=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateNumOps=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateNumOps=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateNumOps=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateNumOps=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateNumOps=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Context=dfs 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalMetadataOperations=0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,317 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumFailedVolumes=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheUsed=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheCapacity=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksCached=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:LastVolumeFailureDate=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Capacity=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Context=FSDatasetState 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data6]'} 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:DfsUsed=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Remaining=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToCache=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToUnCache=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:ReceivedBytes=744936 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeAvgTime=0.009771986970684038 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcSlowCalls=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationSuccesses=27 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumDroppedConnections=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.NumOpenConnectionsPerUser={"jenkins":1} 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcClientBackoff=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeAvgTime=0.006514657980456026 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationFailures=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumOpenConnections=1 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.port=39027 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:SentBytes=627674 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeNumOps=2342 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationSuccesses=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:CallQueueLength=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Context=rpc 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationFailures=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeNumOps=2342 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.serverName=ClientNamenodeProtocol 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeNumOps=2342 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.port=35085 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Context=rpcdetailed 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthNumOps=46 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemMaxM=2200.0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTerminated=0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsWaiting=86 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.SessionId=null 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Context=jvm 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapUsedM=188.845 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTotalExtraSleepTime=30 2023-10-15T16:14:17,318 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Old Generation=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCount=11 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsBlocked=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogWarn=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsRunnable=83 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogInfo=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsNew=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapUsedM=323.47656 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTimedWaiting=119 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapMaxM=2200.0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Young Generation=11 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogError=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.ProcessName=DataNode 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogFatal=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillis=328 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsBatchedInSync=31 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsNumOps=224 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsAvgTime=0.13043478260869565 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportNumOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeNumOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteSnapshotOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsAvgTime=0.03896103896103896 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AllowSnapshotOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:RenameSnapshotOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeAvgTime=0.0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesCreated=80 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesAppended=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportAvgTime=0.0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageAvgTime=0.0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeNumOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeNumOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeNumOps=3 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeAvgTime=0.0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumTimesReReplicationNotScheduled=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SafeModeTime=3 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalNumOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetListingOps=66 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.ProcessName=NameNode 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageNumOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TotalFileOps=540 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSymlinkOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AddBlockOps=28 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DisallowSnapshotOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteFileOps=5 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsNumOps=255 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSnapshotOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PendingEditsCount=1 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ListSnapshottableDirOps=0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockReceivedAndDeletedOps=96 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.SessionId=null 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:14:17,319 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditNumOps=0 2023-10-15T16:14:17,332 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FsImageLoadTime=173 2023-10-15T16:14:17,332 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateFileOps=36 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Context=dfs 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesRenamed=17 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TimeoutReReplications=0 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsQueued=1 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedNumOps=0 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageNumOps=0 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageAvgTime=0.0 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesInGetListingOps=35 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FileInfoOps=270 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetLinkTargetOps=0 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetAdditionalDatanodeOps=0 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsBatched=9 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesDeleted=1 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SuccessfulReReplications=2 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetBlockLocations=118 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesTruncated=0 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SnapshotDiffReportOps=0 2023-10-15T16:14:17,333 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportNumOps=6 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeNumOps=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditAvgTime=0.0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportAvgTime=0.5 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumFailedVolumes=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheCapacity=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheUsed=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:FailedStorageLocations=[Ljava.lang.String;@3db8e0f7 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksCached=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:LastVolumeFailureDate=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Capacity=1054589059072 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToUncache=7 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:DfsUsed=383910 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Remaining=941189184502 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToCache=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpNumOps=5 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWrite=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosNumOps=48 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRemoved=7 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalAvgTime=1.0303030303030303 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsNumOps=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksCached=0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsAvgTime=2.0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DataNodeActiveXceiversCount=1 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalNumOps=200 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsNumOps=51 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,334 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesRead=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpAvgTime=3792.5 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesAvgTime=0.0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionTasks=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncCount=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:NumProcessedCommands=201 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpNumOps=99 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromLocalClient=48 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:VolumeFailures=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosNumOps=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesRead=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosAvgTime=388889.5 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalReadTime=29 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesRead=688752 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsAvgTime=0.75 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromRemoteClient=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksUncached=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DatanodeNetworkErrors=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromLocalClient=99 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksInPendingIBR=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockVerificationFailures=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosNumOps=200 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesWrite=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosNumOps=200 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.SessionId=null 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsNumOps=1 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesWritten=973747 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRead=99 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReplicated=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Context=dfs 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksVerified=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromRemoteClient=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosAvgTime=45642.0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsNumOps=200 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksWritten=48 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsAvgTime=0.9797979797979798 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalWriteTime=22 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcDecodingTimeNanos=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesNumOps=0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosNumOps=1700 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpNumOps=48 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosAvgTime=4914.0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,335 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesWritten=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosAvgTime=9294.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpNumOps=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWrite=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosNumOps=30 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRemoved=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsNumOps=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksCached=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsAvgTime=95.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalNumOps=9 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesRead=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpAvgTime=6809.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesAvgTime=0.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionTasks=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncCount=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:NumProcessedCommands=10 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpNumOps=35 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromLocalClient=28 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:VolumeFailures=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosNumOps=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesRead=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosAvgTime=352052.375 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalReadTime=16 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesRead=75811 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsAvgTime=1.3846153846153846 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromRemoteClient=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksUncached=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DatanodeNetworkErrors=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromLocalClient=35 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksInPendingIBR=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockVerificationFailures=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosNumOps=70 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesWrite=0 2023-10-15T16:14:17,336 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosNumOps=70 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.SessionId=null 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsNumOps=1 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesWritten=620489 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRead=35 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReplicated=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Context=dfs 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksVerified=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromRemoteClient=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosAvgTime=25843.916666666668 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsNumOps=9 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksWritten=28 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsAvgTime=1.0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalWriteTime=12 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcDecodingTimeNanos=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesNumOps=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosNumOps=1285 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpNumOps=28 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosAvgTime=4740.882352941177 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesWritten=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosAvgTime=9895.041666666666 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpAvgTime=0.75 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpNumOps=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpNumOps=4 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWrite=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosNumOps=32 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRemoved=7 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalAvgTime=0.9795918367346939 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsNumOps=0 2023-10-15T16:14:17,337 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksCached=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsAvgTime=2.0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DataNodeActiveXceiversCount=1 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalNumOps=200 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsNumOps=55 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesRead=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpAvgTime=0.0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpAvgTime=3793.25 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesAvgTime=0.0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionTasks=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncCount=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:NumProcessedCommands=201 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpNumOps=89 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromLocalClient=48 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:VolumeFailures=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosNumOps=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesRead=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosAvgTime=304427.5 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalReadTime=12 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesRead=339498 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsAvgTime=0.75 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromRemoteClient=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksUncached=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DatanodeNetworkErrors=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromLocalClient=89 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksInPendingIBR=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockVerificationFailures=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosNumOps=178 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesWrite=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosNumOps=178 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.SessionId=null 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsNumOps=1 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesWritten=973747 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRead=89 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReplicated=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Context=dfs 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksVerified=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromRemoteClient=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosAvgTime=17297.333333333332 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsNumOps=200 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksWritten=48 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsAvgTime=0.9489795918367347 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalWriteTime=28 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcDecodingTimeNanos=0 2023-10-15T16:14:17,338 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesNumOps=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosNumOps=1700 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpNumOps=48 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosAvgTime=4986.545454545455 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesWritten=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosAvgTime=11669.166666666666 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpAvgTime=0.3333333333333333 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpNumOps=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDataNodeMessageCount=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastWrittenTransactionId=255 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:SnapshottableDirectories=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEncryptionZones=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReconstructionBlocks=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptBlocks=0 2023-10-15T16:14:17,339 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastCheckpointTime=1697385845476 2023-10-15T16:14:17,345 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ReadLockLongHoldCount=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumActiveClients=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastLogRoll=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumTimedOutPendingReconstructions=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:VolumeFailuresTotal=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ProvidedCapacityTotal=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingECBlockGroups=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedGB=0.0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecommissioningDataNodes=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.HAState=active 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionBlocks=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyECBlockGroups=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:FilesTotal=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ScheduledReplicationBlocks=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureECBlockGroups=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptECBlockGroups=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MillisSinceLastLoadedEdits=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalECBlockGroups=0 2023-10-15T16:14:17,360 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExcessBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalLoad=32 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExpiredHeartbeats=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInServiceLiveDataNodes=3 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumStaleStorages=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:WriteLockLongHoldCount=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlocksTotal=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:UnderReplicatedBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicatedBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsed=495713 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalReplicatedBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:StaleDataNodes=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumFilesUnderConstruction=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemaining=2816043276636 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PostponedMisreplicatedBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedNonDFS=185085906847 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LockQueueLength=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptReplicatedBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Context=dfs 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomDeadDataNodes=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:Snapshots=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionECBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.TotalSyncTimes= 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReplicationBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotal=3163767177216 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDeadDataNodes=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplOneBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalSyncCount=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastCheckpoint=255 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicationOneBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumLiveDataNodes=3 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlockCapacity=4194304 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotalGB=2946.0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyBlocks=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomLiveDataNodes=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemainingGB=2623.0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Context=regionserver 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheCleared=1 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheHit=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Context=rpc 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheUpdated=65 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemMaxM=2200.0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTerminated=0 2023-10-15T16:14:17,361 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsWaiting=86 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.SessionId= 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Context=jvm 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapUsedM=188.84634 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Old Generation=0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCount=11 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsBlocked=0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogWarn=0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsRunnable=83 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogInfo=0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsNew=0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapUsedM=324.47656 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTimedWaiting=119 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapMaxM=2200.0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Young Generation=11 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogError=0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.ProcessName=Master 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogFatal=0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillis=328 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsCount=0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeTotal=0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointPercentComplete=1.0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageTotal=1 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsTotal=0 2023-10-15T16:14:17,362 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointCount=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:ElapsedTime=681 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsPercentComplete=1.0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsElapsedTime=7 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointElapsedTime=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModePercentComplete=1.0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:PercentComplete=1.0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointTotal=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeCount=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeElapsedTime=13 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageElapsedTime=62 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageCount=1 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImagePercentComplete=1.0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumFailedVolumes=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheCapacity=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheUsed=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:FailedStorageLocations=[Ljava.lang.String;@337f0930 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksCached=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:LastVolumeFailureDate=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Capacity=1054589059072 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToUncache=7 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:DfsUsed=382860 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Remaining=941188783094 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToCache=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsCount=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeTotal=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointPercentComplete=1.0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageTotal=1 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsTotal=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointCount=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:ElapsedTime=681 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsPercentComplete=1.0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsElapsedTime=7 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointElapsedTime=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModePercentComplete=1.0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:PercentComplete=1.0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointTotal=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeCount=0 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeElapsedTime=13 2023-10-15T16:14:17,364 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageElapsedTime=62 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageCount=1 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImagePercentComplete=1.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RackLocalityCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionCountSkewCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaHostCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ServerLocalityCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MemStoreSizeCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_95th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_Overall=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ReadRequestCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_25th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_98th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_min=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Context=master 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_WriteRequestCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaRackCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_PrimaryRegionCountSkewCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:miscInvocationCount=18 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_median=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_num_ops=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_75th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MoveCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_max=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_mean=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_CPRequestCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_TableSkewCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_StoreFileCostFunction=0.0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_90th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.isBalancerActive=true 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99.9th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_95th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99.9th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_median=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_mean=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_25th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_max=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_95th_percentile=0 2023-10-15T16:14:17,365 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_num_ops=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_25th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_98th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_min=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_mean=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_90th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_75th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_90th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_mean=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_num_ops=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_95th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_min=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_median=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_num_ops=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_mean=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_98th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_75th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_median=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99.9th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Context=regionserver 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_max=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_num_ops=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_num_ops=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_min=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_90th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_mean=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_95th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_max=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_75th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_75th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_min=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_num_ops=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_75th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_max=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_min=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_98th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_90th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_max=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_98th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_90th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_75th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_95th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_25th_percentile=0 2023-10-15T16:14:17,366 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_max=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_max=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_98th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_num_ops=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99.9th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_min=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_98th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_95th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_min=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_median=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:BlockBytesScannedCount=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99.9th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_25th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_median=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_90th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_95th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_num_ops=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_max=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_mean=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_max=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_min=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_mean=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_95th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99.9th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_25th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99.9th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_90th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_mean=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_95th_percentile=0 2023-10-15T16:14:17,367 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_min=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_75th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_max=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_max=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_90th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_max=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_98th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_25th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_median=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_median=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_75th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_mean=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99.9th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_median=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_90th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_mean=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_mean=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_75th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_num_ops=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99.9th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_median=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_75th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_median=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_median=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_98th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_95th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_75th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_95th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_num_ops=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_max=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_98th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_25th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_95th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99.9th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_num_ops=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_25th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99.9th_percentile=0 2023-10-15T16:14:17,375 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_median=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_90th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_25th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_median=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_min=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_max=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_90th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_max=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_75th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_25th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_98th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_min=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_mean=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99.9th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_25th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_min=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_mean=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_min=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_98th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_min=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_98th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99.9th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_median=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_num_ops=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_median=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_90th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_min=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_25th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,376 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsCount=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_98th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_median=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestRatePerSecond=0.0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerMissCount=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_mean=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_median=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_98th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_95th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_median=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_90th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_5min_rate=0.0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_min=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_95th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitRatio=0.0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_75th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_90th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_num_ops=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_min=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_25th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestCount=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedOutputBytes=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_90th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkHitCount=1 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadOpsInCurThread=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_median=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestRatePerSecond=0.0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_25th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_min=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_num_ops=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_num_ops=26 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_90th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockBytesScannedCount=4389 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_max=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_75th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_max=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_mean=0 2023-10-15T16:14:17,377 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_num_ops=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_25th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheEvictionCount=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99.9th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheCount=3 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:smallCompactionQueueLength=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCount=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_min=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_num_ops=15 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_mean=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowIncrementCount=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushQueueLength=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_mean=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_75th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:localBytesRead=108773 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushCount=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_median=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutateFailedCount=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_max=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorPoolAllocationBytes=1530880 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_98th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_min=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_median=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_min=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:updatesBlockedTime=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_median=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_mean=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitQueueLength=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_min=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_min=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_95th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCachingCount=3 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_25th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99.9th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileAge=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_median=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_75th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_max=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_95th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_num_ops=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_min=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cpRequestCount=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_90th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_median=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_max=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99th_percentile=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_num_ops=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALCount=0 2023-10-15T16:14:17,378 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_98th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitRatio=0.7272727272727273 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataMissCount=1 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestCount=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissCount=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_75th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_mean=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheEvictionCount=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheFreeSize=922067336 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_95th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_median=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_75th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_max=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_median=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:activeScanners=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissCount=3 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_mean=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_max=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_max=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_95th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactionQueueLength=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_95th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_num_ops=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_max=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_1min_rate=0.0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_min=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_median=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_num_ops=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_median=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_mean=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_75th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_min=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99.9th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_median=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_25th_percentile=0 2023-10-15T16:14:17,379 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_25th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_90th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_num_ops=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheSize=679544 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_num_ops=6 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_num_ops=26 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_98th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_count=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_median=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowGetCount=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_median=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterRequestsCount=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_min=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_min=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_25th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_max=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileSize=414534 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_min=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileCount=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsCount=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99.9th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,380 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_98th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_num_ops=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedOutputBytes=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_max=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_num_ops=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_max=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticBloomSize=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexMissCount=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseInfoThresholdExceeded=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCount=3 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCount=3 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_max=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedToMob=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedMemstoreBytes=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheExpressHitPercent=72.72727272727273 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionServerStartTime=1697385859285 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedInputBytes=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedFromMob=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_max=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRequestCount=47 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_25th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcGetRequestCount=3 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_90th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeCount=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCountHitPercent=72.72727272727273 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_max=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_mean=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_min=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_98th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_min=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsSize=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_mean=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowDeleteCount=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockedRequestCount=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_mean=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheAccessCount=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_min=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_median=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_min=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_90th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_min=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_95th_percentile=0 2023-10-15T16:14:17,382 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_90th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheEvictedCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_90th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_max=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:avgStoreFileAge=1819 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_25th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitRequestCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_num_ops=1 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_min=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalBytesRead=108773 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedToMob=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_max=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedInputBytes=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheSize=679544 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileCount=473 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_mean=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcScanRequestCount=26 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreOffHeapSize=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_75th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheMissCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowPutCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_max=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreSize=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_75th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsSize=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_mean=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_min=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_75th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_median=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_98th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_median=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Context=regionserver 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:scannerLeaseExpiredCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:minStoreFileAge=1782 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_98th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcFullScanRequestCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_mean=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitSuccessCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_mean=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_75th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_25th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCountPrimary=3 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_max=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataBlockCount=1 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_98th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_max=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_median=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_mean=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCachingCount=8 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFailedInsertionCount=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_90th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsSize=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean_rate=0.0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticIndexSize=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorUsedBufferCount=2 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_min=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99.9th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissRatio=0.2727272727272727 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_75th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_max=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_mean=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_95th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,383 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMultiRequestCount=1 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheSize=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexHitCount=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_max=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_98th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedOutputBytes=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocal=0.0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_98th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_max=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_num_ops=3 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99.9th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_25th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_min=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_mean=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_15min_rate=0.0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99.9th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_min=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_95th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_75th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_median=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_median=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_mean=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_95th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitCount=8 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:largeCompactionQueueLength=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_mean=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutatePassedCount=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_max=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_95th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissRatio=0.0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsCount=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_min=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_mean=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_min=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:shortCircuitBytesRead=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99.9th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:17,384 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_median=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_min=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoMissCount=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileIndexSize=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99.9th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_mean=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerHitCount=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_98th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsSize=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCountPrimary=8 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_min=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_max=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99.9th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_90th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_90th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_num_ops=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_max=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_mean=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_max=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_min=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRowActionRequestCount=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMutateRequestCount=16 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheCount=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_25th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_mean=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataHitCount=7 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:zeroCopyBytesRead=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_98th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionCount=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_num_ops=1 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileCount=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_num_ops=1 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_median=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALSize=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_num_ops=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_mean=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_median=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheFreeSize=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:averageRegionSize=17474 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_95th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaMissCount=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:17,385 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_75th_percentile=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:filteredReadRequestCount=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaHitCount=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_75th_percentile=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_num_ops=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_95th_percentile=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_median=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCount=8 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_num_ops=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.serverName=8c5d72df62de,45999,1697385859285 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReads=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowAppendCount=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:numReferenceFiles=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedFromMob=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSize=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_max=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_90th_percentile=0 2023-10-15T16:14:17,386 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_median=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_mean=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_98th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadWins=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseWarnThresholdExceeded=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_max=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_25th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoHitCount=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_median=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreHeapSize=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_25th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_min=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99.9th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_25th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_num_ops=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_90th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_max=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99.9th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_max=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_median=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsSize=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_median=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,388 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_median=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_min=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkMissCount=2 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_95th_percentile=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFreeSize=922067336 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheHitPercent=0.0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemMaxM=2200.0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTerminated=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsWaiting=86 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.SessionId=null 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Context=jvm 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapUsedM=188.85072 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTotalExtraSleepTime=30 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Old Generation=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCount=11 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsBlocked=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogWarn=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsRunnable=83 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogInfo=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsNew=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapUsedM=325.47656 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTimedWaiting=119 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapMaxM=2200.0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Young Generation=11 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogError=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.ProcessName=NameNode 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogFatal=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillis=328 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemMaxM=2200.0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTerminated=0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsWaiting=86 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.SessionId=null 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Context=jvm 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapUsedM=188.85118 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTotalExtraSleepTime=40 2023-10-15T16:14:17,389 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Old Generation=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCount=11 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsBlocked=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogWarn=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsRunnable=83 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogInfo=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsNew=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapUsedM=325.47656 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTimedWaiting=119 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapMaxM=2200.0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Young Generation=11 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogError=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.ProcessName=DataNode 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogFatal=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillis=328 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_90th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_25th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_num_ops=14 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99.9th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_90th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_95th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_median=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_max=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_95th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_98th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_98th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_25th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:fsChecksumFailureCount=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_median=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_75th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_min=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Context=regionserver 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_max=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_min=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_90th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_75th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_max=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99.9th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_98th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_mean=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_95th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_num_ops=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_min=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_median=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99.9th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_mean=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_num_ops=186 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_25th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_mean=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_75th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99th_percentile=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount=0 2023-10-15T16:14:17,390 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context=dfs 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.1=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.2=dfs 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.2=8c5d72df62de 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.1=dfs 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.1=8c5d72df62de 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.2=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Context=master 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:numMasterWALs=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsCount=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_98th_percentile=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_median=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestRatePerSecond=0.0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerMissCount=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_mean=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99th_percentile=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_median=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_98th_percentile=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_95th_percentile=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_median=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_90th_percentile=0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_5min_rate=0.0 2023-10-15T16:14:17,391 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_min=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_95th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitRatio=0.0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_75th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_90th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_num_ops=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_min=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_25th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestCount=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedOutputBytes=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_90th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkHitCount=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadOpsInCurThread=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_median=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestRatePerSecond=0.0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_25th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_min=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_num_ops=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_num_ops=2 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_90th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockBytesScannedCount=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_max=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_75th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_max=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_mean=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_num_ops=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_25th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheEvictionCount=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99.9th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheCount=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:smallCompactionQueueLength=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCount=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_min=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_num_ops=6 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_mean=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowIncrementCount=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushQueueLength=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_mean=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_75th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:localBytesRead=108773 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushCount=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_median=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutateFailedCount=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_max=0 2023-10-15T16:14:17,392 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorPoolAllocationBytes=199680 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_98th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_min=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_median=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_min=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:updatesBlockedTime=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_median=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_mean=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitQueueLength=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_min=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_min=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_95th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCachingCount=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_25th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99.9th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileAge=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_median=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_75th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_max=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_95th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_num_ops=1 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_min=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cpRequestCount=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_90th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_median=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_max=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_num_ops=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALCount=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_98th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitRatio=0.0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataMissCount=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestCount=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissCount=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_75th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_mean=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheEvictionCount=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheFreeSize=922070024 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_95th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_median=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_75th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_num_ops=1 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_max=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_median=0 2023-10-15T16:14:17,393 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:activeScanners=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissCount=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_mean=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_max=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_max=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_95th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactionQueueLength=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_95th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_num_ops=1 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_max=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_1min_rate=0.0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_min=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_median=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_num_ops=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_median=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_mean=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_75th_percentile=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_min=0 2023-10-15T16:14:17,394 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99.9th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_median=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_25th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_25th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_90th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_num_ops=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheSize=676856 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_num_ops=2 2023-10-15T16:14:17,412 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_98th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_count=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_median=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowGetCount=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_median=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterRequestsCount=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_min=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_min=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_25th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_max=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileSize=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_min=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileCount=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsCount=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99.9th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_98th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_num_ops=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedOutputBytes=5860 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_max=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_num_ops=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_max=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticBloomSize=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexMissCount=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseInfoThresholdExceeded=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCount=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCount=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_max=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedToMob=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedMemstoreBytes=928 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheExpressHitPercent=0.0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionServerStartTime=1697385851561 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedInputBytes=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedFromMob=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_max=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRequestCount=11 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_25th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcGetRequestCount=3 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_90th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeCount=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCountHitPercent=0.0 2023-10-15T16:14:17,413 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_max=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_mean=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_min=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_98th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_min=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsSize=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_mean=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowDeleteCount=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockedRequestCount=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_mean=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheAccessCount=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_min=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_median=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_min=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_90th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_min=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_95th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_90th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheEvictedCount=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_90th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_max=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:avgStoreFileAge=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_25th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheCount=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitRequestCount=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_min=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalBytesRead=108773 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedToMob=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_max=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedInputBytes=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheSize=676856 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileCount=239 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_mean=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcScanRequestCount=2 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreOffHeapSize=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_75th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheMissCount=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowPutCount=0 2023-10-15T16:14:17,414 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_max=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreSize=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_75th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsSize=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_mean=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_min=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_75th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_median=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_98th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_median=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Context=regionserver 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:scannerLeaseExpiredCount=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:minStoreFileAge=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitCount=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_98th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcFullScanRequestCount=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_mean=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitSuccessCount=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_mean=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_75th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_25th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCountPrimary=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_max=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataBlockCount=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_98th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_max=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_median=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsCount=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_mean=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsCount=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCachingCount=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFailedInsertionCount=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_90th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsSize=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean_rate=0.0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticIndexSize=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_min=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99.9th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissRatio=0.0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_75th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_max=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_mean=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:17,415 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_95th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMultiRequestCount=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheSize=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexHitCount=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_max=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_98th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedOutputBytes=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocal=0.0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_98th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_max=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_num_ops=3 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99.9th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_25th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_min=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_mean=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_15min_rate=0.0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99.9th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_min=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_95th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_75th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_median=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_median=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_mean=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_95th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitCount=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:largeCompactionQueueLength=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_mean=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutatePassedCount=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_max=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_95th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissRatio=0.0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsCount=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_min=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_mean=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_min=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:shortCircuitBytesRead=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99.9th_percentile=0 2023-10-15T16:14:17,416 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_median=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_min=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoMissCount=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileIndexSize=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99.9th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_mean=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerHitCount=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_98th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsSize=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCountPrimary=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_min=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_max=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99.9th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_90th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_90th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_num_ops=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_max=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_mean=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_max=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_min=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRowActionRequestCount=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMutateRequestCount=6 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheCount=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_25th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:17,417 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_mean=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataHitCount=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:zeroCopyBytesRead=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_98th_percentile=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionCount=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_num_ops=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileCount=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_num_ops=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_median=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALSize=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_num_ops=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_mean=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_median=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheFreeSize=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:averageRegionSize=928 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_95th_percentile=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaMissCount=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_75th_percentile=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:filteredReadRequestCount=0 2023-10-15T16:14:17,422 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaHitCount=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_75th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_num_ops=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_95th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_median=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCount=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_num_ops=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.serverName=8c5d72df62de,34443,1697385851561 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReads=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowAppendCount=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:numReferenceFiles=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedFromMob=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSize=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_max=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_90th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_median=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_mean=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_98th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadWins=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseWarnThresholdExceeded=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_max=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_25th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoHitCount=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_median=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreHeapSize=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_25th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_min=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99.9th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_25th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_num_ops=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_90th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_max=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99.9th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_max=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_median=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsSize=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_median=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_median=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_min=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkMissCount=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,423 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_95th_percentile=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFreeSize=922070024 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheHitPercent=0.0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsCount=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_98th_percentile=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_median=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestRatePerSecond=0.0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerMissCount=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_mean=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99th_percentile=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_median=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_98th_percentile=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_95th_percentile=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_median=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_90th_percentile=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_5min_rate=0.0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_min=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_95th_percentile=0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitRatio=0.0 2023-10-15T16:14:17,424 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_75th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_90th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_num_ops=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_min=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_25th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestCount=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedOutputBytes=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_90th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkHitCount=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadOpsInCurThread=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_median=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestRatePerSecond=0.0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_25th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_min=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_num_ops=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_num_ops=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_90th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockBytesScannedCount=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_max=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_75th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_max=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_mean=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_num_ops=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_25th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheEvictionCount=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99.9th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheCount=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:smallCompactionQueueLength=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCount=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_min=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_num_ops=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_mean=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowIncrementCount=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushQueueLength=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_mean=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_75th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:localBytesRead=108773 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushCount=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_median=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutateFailedCount=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_max=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorPoolAllocationBytes=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_98th_percentile=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_min=0 2023-10-15T16:14:17,425 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_median=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_min=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:updatesBlockedTime=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_median=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_mean=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitQueueLength=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_min=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_min=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_95th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCachingCount=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_25th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99.9th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileAge=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_median=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_75th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_max=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_95th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_num_ops=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_min=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cpRequestCount=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_90th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_median=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_max=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_num_ops=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALCount=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_98th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitRatio=0.0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataMissCount=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestCount=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissCount=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_75th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_mean=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheEvictionCount=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheFreeSize=922070024 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_95th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_median=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_75th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_max=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_median=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:activeScanners=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissCount=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_mean=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:17,426 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_max=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_max=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_95th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactionQueueLength=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_95th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_num_ops=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_max=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_1min_rate=0.0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_min=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_median=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_num_ops=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_median=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_mean=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_75th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_min=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99.9th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_median=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_25th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_25th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_90th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_num_ops=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheSize=676856 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_num_ops=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_98th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_count=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_median=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowGetCount=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_median=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterRequestsCount=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_min=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_min=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_25th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_max=0 2023-10-15T16:14:17,427 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileSize=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_min=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileCount=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99th_percentile=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsCount=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99.9th_percentile=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_98th_percentile=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_num_ops=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedOutputBytes=0 2023-10-15T16:14:17,428 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_max=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_num_ops=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_max=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticBloomSize=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexMissCount=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseInfoThresholdExceeded=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCount=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCount=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_max=0 2023-10-15T16:14:17,441 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedToMob=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedMemstoreBytes=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheExpressHitPercent=0.0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionServerStartTime=1697385851811 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedInputBytes=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedFromMob=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_max=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRequestCount=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_25th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcGetRequestCount=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_90th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeCount=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCountHitPercent=0.0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_max=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_mean=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_min=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_98th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_min=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsSize=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_mean=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowDeleteCount=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockedRequestCount=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_mean=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheAccessCount=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_min=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_median=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_min=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_90th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_min=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_95th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_90th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheEvictedCount=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_90th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_max=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:avgStoreFileAge=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_25th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheCount=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitRequestCount=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_min=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalBytesRead=108773 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedToMob=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_max=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedInputBytes=0 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheSize=676856 2023-10-15T16:14:17,442 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileCount=120 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_mean=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcScanRequestCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreOffHeapSize=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_75th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheMissCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowPutCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_max=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreSize=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_75th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsSize=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_mean=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_min=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_75th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_median=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_98th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_median=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Context=regionserver 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:scannerLeaseExpiredCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:minStoreFileAge=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_98th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcFullScanRequestCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_mean=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitSuccessCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_mean=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_75th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_25th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCountPrimary=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_max=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataBlockCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_98th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_max=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_median=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_mean=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCachingCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFailedInsertionCount=0 2023-10-15T16:14:17,443 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_90th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsSize=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean_rate=0.0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticIndexSize=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorUsedBufferCount=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_min=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99.9th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissRatio=0.0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_75th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_max=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_mean=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_95th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMultiRequestCount=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheSize=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexHitCount=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_max=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_98th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedOutputBytes=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocal=0.0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_98th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_max=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_num_ops=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99.9th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_25th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_min=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_mean=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_15min_rate=0.0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99.9th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_min=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_95th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_75th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_median=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_median=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_mean=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_95th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitCount=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:largeCompactionQueueLength=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_mean=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:17,444 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutatePassedCount=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_max=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_95th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissRatio=0.0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsCount=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_min=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_mean=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_min=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:shortCircuitBytesRead=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99.9th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_median=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_min=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoMissCount=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileIndexSize=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99.9th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_mean=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerHitCount=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_98th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsSize=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCountPrimary=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_min=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_max=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99.9th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_90th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_90th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_num_ops=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_max=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_mean=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_max=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_min=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRowActionRequestCount=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMutateRequestCount=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheCount=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_25th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationBytes=87783 2023-10-15T16:14:17,445 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_mean=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataHitCount=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:zeroCopyBytesRead=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_98th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionCount=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_num_ops=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileCount=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_num_ops=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_median=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALSize=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_num_ops=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_mean=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_median=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheFreeSize=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:averageRegionSize=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_95th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaMissCount=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_75th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:filteredReadRequestCount=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaHitCount=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_75th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_num_ops=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_95th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_median=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCount=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_num_ops=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.serverName=8c5d72df62de,40585,1697385851811 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReads=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowAppendCount=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:numReferenceFiles=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedFromMob=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSize=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_max=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_90th_percentile=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_median=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_mean=0 2023-10-15T16:14:17,446 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_98th_percentile=0 2023-10-15T16:14:17,447 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadWins=0 2023-10-15T16:14:17,447 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseWarnThresholdExceeded=0 2023-10-15T16:14:17,447 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,447 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,447 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_max=0 2023-10-15T16:14:17,447 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_25th_percentile=0 2023-10-15T16:14:17,447 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoHitCount=0 2023-10-15T16:14:17,447 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_median=0 2023-10-15T16:14:17,447 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreHeapSize=0 2023-10-15T16:14:17,447 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_25th_percentile=0 2023-10-15T16:14:17,447 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_min=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99.9th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_25th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_num_ops=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_90th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_max=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99.9th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_max=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_median=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsSize=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_median=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_median=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_min=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkMissCount=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_95th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFreeSize=922070024 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheHitPercent=0.0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:17,448 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsCount=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_98th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_median=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestRatePerSecond=0.0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerMissCount=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_mean=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_median=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_98th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_95th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_median=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_90th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_5min_rate=0.0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_min=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_95th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitRatio=0.0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_75th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_90th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_mean=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_90th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_num_ops=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_min=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_25th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestCount=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedOutputBytes=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_90th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkHitCount=1 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadOpsInCurThread=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_median=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestRatePerSecond=0.0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_25th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_min=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_num_ops=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_num_ops=11 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_90th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_98th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockBytesScannedCount=6840 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_max=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_75th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCountPrimary=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_max=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_mean=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_num_ops=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_25th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_75th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheEvictionCount=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99.9th_percentile=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheCount=4 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:smallCompactionQueueLength=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCount=0 2023-10-15T16:14:17,449 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_min=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_num_ops=7 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_num_ops=1 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_mean=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowIncrementCount=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushQueueLength=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_mean=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_75th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:localBytesRead=108773 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushCount=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_median=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutateFailedCount=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_max=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorPoolAllocationBytes=599040 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_98th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_median=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_min=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_median=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_min=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:updatesBlockedTime=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_median=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_mean=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitQueueLength=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_min=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_min=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99.9th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_mean=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_95th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCachingCount=4 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_25th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99.9th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileAge=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_median=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_75th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_max=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_95th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_num_ops=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_min=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cpRequestCount=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_median=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_90th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_median=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_95th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_max=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_num_ops=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALCount=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_98th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitRatio=0.72 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataMissCount=6 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestCount=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,450 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissCount=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_75th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_mean=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheEvictionCount=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheFreeSize=922061632 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_95th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_max=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_median=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_75th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_num_ops=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_max=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_median=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:activeScanners=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterNegativeResultsCount=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissCount=7 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_mean=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_max=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_max=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_95th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactionQueueLength=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_95th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_num_ops=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_mean=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_max=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_25th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_1min_rate=0.0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_min=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_median=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_num_ops=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_median=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_mean=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_75th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_min=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99.9th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_median=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_25th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_25th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_90th_percentile=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_num_ops=0 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheSize=685248 2023-10-15T16:14:17,451 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_num_ops=7 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_min=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_num_ops=11 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_98th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_max=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_count=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_median=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowGetCount=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_median=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterRequestsCount=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_min=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_min=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_90th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_25th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSizeGrowthRate=0.0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_num_ops=1 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_max=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileSize=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_min=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileCount=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsCount=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99.9th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_98th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_num_ops=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedOutputBytes=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99.9th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_max=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_num_ops=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_max=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticBloomSize=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexMissCount=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseInfoThresholdExceeded=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCount=4 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCount=7 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_max=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedToMob=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedMemstoreBytes=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheExpressHitPercent=81.81818181818183 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionServerStartTime=1697385888915 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedInputBytes=18105 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedFromMob=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_max=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRequestCount=21 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_25th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcGetRequestCount=1 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_90th_percentile=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeCount=0 2023-10-15T16:14:17,452 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCountHitPercent=72.0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_max=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_mean=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_min=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_98th_percentile=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_min=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsSize=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_mean=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_75th_percentile=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowDeleteCount=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockedRequestCount=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_mean=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheAccessCount=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_min=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_median=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_min=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_90th_percentile=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99th_percentile=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_min=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_25th_percentile=0 2023-10-15T16:14:17,453 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_95th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_90th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheEvictedCount=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_90th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_max=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:avgStoreFileAge=12090 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_25th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheCount=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitRequestCount=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_num_ops=1 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_min=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalBytesRead=108773 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedToMob=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_max=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_max=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_95th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedInputBytes=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheSize=685248 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileCount=339 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_mean=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcScanRequestCount=11 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreOffHeapSize=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_75th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_mean=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheMissCount=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexMissCount=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowPutCount=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_98th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_max=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreSize=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_75th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsSize=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_mean=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_min=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_75th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_median=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_98th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_median=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Context=regionserver 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:scannerLeaseExpiredCount=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:minStoreFileAge=2959 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitCount=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_98th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,468 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcFullScanRequestCount=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_90th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_75th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_mean=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitSuccessCount=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_mean=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_min=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_75th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_25th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCountPrimary=7 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_max=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataBlockCount=3 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_98th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_max=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_median=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsCount=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_mean=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsCount=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCachingCount=18 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_75th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFailedInsertionCount=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_90th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsSize=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean_rate=0.0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticIndexSize=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_95th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_min=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99.9th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissRatio=0.28 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_75th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_max=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_mean=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_num_ops=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_95th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMultiRequestCount=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheSize=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexHitCount=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_max=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_98th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedOutputBytes=5921 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocal=0.0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_98th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_max=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_num_ops=1 2023-10-15T16:14:17,469 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99.9th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_25th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_min=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_mean=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_15min_rate=0.0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_25th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99.9th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_min=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_95th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_75th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_median=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_median=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_mean=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_95th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitCount=18 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_95th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:largeCompactionQueueLength=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_mean=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_median=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutatePassedCount=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_max=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_95th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_90th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissRatio=0.0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsCount=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_25th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_min=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_mean=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_min=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:shortCircuitBytesRead=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_75th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99.9th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_95th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_median=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_98th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_min=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_mean=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_min=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoMissCount=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileIndexSize=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99.9th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:14:17,470 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_mean=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerHitCount=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_98th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsSize=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCountPrimary=18 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_min=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_max=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99.9th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_90th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_90th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_90th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_num_ops=1 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_max=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_mean=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_max=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_min=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRowActionRequestCount=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMutateRequestCount=8 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheCount=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_25th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationBytes=89913 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_mean=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataHitCount=17 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:zeroCopyBytesRead=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_98th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionCount=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_num_ops=1 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileCount=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_num_ops=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_median=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALSize=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_num_ops=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_mean=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_median=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheFreeSize=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:averageRegionSize=32082 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_95th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaMissCount=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_mean=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_98th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_75th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:filteredReadRequestCount=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaHitCount=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_75th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexHitCount=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_num_ops=1 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_95th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:14:17,471 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_median=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCount=18 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_num_ops=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.serverName=8c5d72df62de,35419,1697385888915 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReads=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowAppendCount=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:numReferenceFiles=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedFromMob=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSize=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_max=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_90th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_median=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_mean=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_98th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadWins=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseWarnThresholdExceeded=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_max=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_25th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoHitCount=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_median=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreHeapSize=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_25th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_min=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99.9th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_25th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_num_ops=1 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_90th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_max=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99.9th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_max=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_median=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsSize=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_median=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_median=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_min=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkMissCount=1 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_95th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFreeSize=922061632 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheHitPercent=0.0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterEligibleRequestsCount=0 2023-10-15T16:14:17,472 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_num_ops=0 2023-10-15T16:14:17,473 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_98th_percentile=0 2023-10-15T16:14:17,473 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_num_ops=2 2023-10-15T16:14:17,473 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99th_percentile=0 2023-10-15T16:14:17,473 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.walReaderEditsBufferUsage=80117 2023-10-15T16:14:17,473 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_25th_percentile=0 2023-10-15T16:14:17,473 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfHFileRefsQueue=0 2023-10-15T16:14:17,473 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logReadInBytes=83847 2023-10-15T16:14:17,473 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_75th_percentile=0 2023-10-15T16:14:17,473 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfLogQueue=-3 2023-10-15T16:14:17,473 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_90th_percentile=0 2023-10-15T16:14:17,473 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99.9th_percentile=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_num_ops=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedOps=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_min=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedHFiles=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_min=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_mean=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBytes=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_90th_percentile=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedOps=1000 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedBatches=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_98th_percentile=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_max=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_95th_percentile=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedBatches=1 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_mean=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_95th_percentile=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedHFiles=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_median=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.uncleanlyClosedLogs=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99th_percentile=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.closedLogsWithUnknownFileLength=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_max=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.repeatedLogFileBytes=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedRecoverQueues=5 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Context=regionserver 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedRecoverQueues=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.restartedLogReading=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.ignoredUncleanlyClosedLogContentsInBytes=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsRead=40 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.numInitializing=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99.9th_percentile=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_75th_percentile=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsFiltered=38 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_25th_percentile=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedLogs=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.failedBatches=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBatches=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_median=0 2023-10-15T16:14:17,474 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsAvgTime=0.0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpNumOps=1 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWriteFallback=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpNumOps=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWrite=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersisted=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosNumOps=26 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksDeletedInPendingIBR=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRemoved=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsNumOps=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksCached=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsAvgTime=94.0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DataNodeActiveXceiversCount=-2 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalNumOps=9 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpAvgTime=0.0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsNumOps=32 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesLazyPersisted=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesRead=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpAvgTime=6326.928571428572 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesAvgTime=0.0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionTasks=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncCount=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:NumProcessedCommands=10 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpNumOps=27 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromLocalClient=28 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:VolumeFailures=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcInvalidReconstructionTasks=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosNumOps=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesRead=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksReadHits=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosAvgTime=561936.0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalReadTime=9 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesRead=52810 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsAvgTime=1.7142857142857142 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromRemoteClient=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksUncached=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DatanodeNetworkErrors=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromLocalClient=27 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksInPendingIBR=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockVerificationFailures=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpAvgTime=0.0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosNumOps=54 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesWrite=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosNumOps=54 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.SessionId=null 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsNumOps=1 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivedInPendingIBR=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesWritten=620489 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionRemoteBytesRead=0 2023-10-15T16:14:17,475 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcFailedReconstructionTasks=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRead=27 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReplicated=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Context=dfs 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksVerified=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromRemoteClient=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksGetLocalPathInfo=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosAvgTime=20303.65 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpNumOps=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsNumOps=9 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksWritten=28 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsAvgTime=1.0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SumOfActorCommandQueueLength=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalWriteTime=12 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvicted=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcDecodingTimeNanos=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionReadTimeMillis=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesNumOps=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosNumOps=1285 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpNumOps=28 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosAvgTime=23635.714285714286 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesWritten=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosAvgTime=0.0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesWritten=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosAvgTime=10588.8 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpAvgTime=1.0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpNumOps=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionWriteTimeMillis=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivingInPendingIBR=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumFailedVolumes=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheCapacity=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheUsed=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:FailedStorageLocations=[Ljava.lang.String;@37408b87 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksCached=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:LastVolumeFailureDate=0 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Capacity=1054589059072 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToUncache=7 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:DfsUsed=625022 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Remaining=941187013622 2023-10-15T16:14:17,476 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToCache=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numRegionSizeReports=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_min=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_num_ops=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_98th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_num_ops=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numNamespaceInQuotaViolation=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_75th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_95th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_90th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_median=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_max=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.namespaceSpaceQuotaOverview=[] 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numTablesInQuotaViolation=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_num_ops=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99.9th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_max=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_98th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_median=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_min=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_max=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Context=master 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99.9th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_95th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_median=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_mean=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_mean=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_max=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_25th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_mean=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_min=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_75th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_mean=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_90th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_min=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_median=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.tableSpaceQuotaOverview=[] 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numSpaceQuotas=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_25th_percentile=0 2023-10-15T16:14:17,477 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_num_ops=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenSubmittedCount=14 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignSubmittedCount=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_num_ops=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99.9th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_median=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_max=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_95th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_95th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_min=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_mean=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_min=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_median=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_min=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveSubmittedCount=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_25th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_mean=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99.9th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_max=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnFS=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerRegions=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_75th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_95th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignSubmittedCount=13 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_75th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_98th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_num_ops=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_90th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignFailedCount=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_num_ops=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_90th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignFailedCount=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_25th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_98th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_98th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveFailedCount=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_mean=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_75th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeFailedCount=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Context=master 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_num_ops=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseSubmittedCount=1 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:holes=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_75th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99.9th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_95th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_95th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeSubmittedCount=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99.9th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_median=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99.9th_percentile=0 2023-10-15T16:14:17,478 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_mean=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCount=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenSubmittedCount=1 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseFailedCount=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_max=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:inconsistentRegions=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_90th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenFailedCount=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnRS=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99.9th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_75th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_75th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_min=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_25th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerOpenRegions=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_95th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_25th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:overlaps=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenFailedCount=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_90th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_98th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitFailedCount=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_max=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_98th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_mean=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_num_ops=13 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_median=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_mean=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCountOverThreshold=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_min=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_75th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_90th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_num_ops=1 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_75th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_95th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_25th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_median=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_25th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitSubmittedCount=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_mean=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritOldestAge=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:emptyRegionInfoRegions=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_max=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_25th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99.9th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_90th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_min=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99.9th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_num_ops=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_max=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_95th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_median=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_max=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_90th_percentile=0 2023-10-15T16:14:17,479 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_25th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_25th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_min=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_90th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_95th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_max=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_num_ops=1 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_min=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:operationCount=15 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_98th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:deadServerOpenRegions=3 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_mean=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_median=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_min=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99.9th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_98th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_median=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_max=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_mean=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_num_ops=14 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_98th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_90th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_median=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_98th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_75th_percentile=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumFailedVolumes=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheUsed=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheCapacity=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksCached=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:LastVolumeFailureDate=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Capacity=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Context=FSDatasetState 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data2]'} 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:DfsUsed=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Remaining=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToCache=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToUnCache=0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,488 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalDataFileIos=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalFileIoErrors=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateNumOps=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateNumOps=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateNumOps=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateNumOps=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateNumOps=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateNumOps=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Context=dfs 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalMetadataOperations=0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsNumOps=17 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportNumOps=14 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsNumOps=54 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoAvgTime=0.6666666666666666 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoNumOps=675 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestNumOps=3 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedNumOps=159 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateNumOps=60 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.port=39027 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeNumOps=3 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportAvgTime=0.3333333333333333 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Context=rpcdetailed 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameNumOps=41 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportAvgTime=0.21428571428571427 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateAvgTime=5.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteNumOps=60 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingNumOps=206 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsNumOps=322 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportNumOps=3 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeAvgTime=0.6666666666666666 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateAvgTime=0.0 2023-10-15T16:14:17,489 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockAvgTime=0.0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionNumOps=1 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockNumOps=49 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteNumOps=33 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateNumOps=1 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesNumOps=16 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseAvgTime=0.0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsAvgTime=0.0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeNumOps=1 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatNumOps=600 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseNumOps=19 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseNumOps=5 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseAvgTime=0.0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatAvgTime=0.0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemMaxM=2200.0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTerminated=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsWaiting=86 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.SessionId=null 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Context=jvm 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapUsedM=188.86484 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTotalExtraSleepTime=55 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Old Generation=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCount=11 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsBlocked=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogWarn=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsRunnable=83 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogInfo=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsNew=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapUsedM=327.47656 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTimedWaiting=119 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapMaxM=2200.0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Young Generation=11 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogError=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.ProcessName=DataNode 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogFatal=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillis=328 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDataNodeMessageCount=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastWrittenTransactionId=431 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:SnapshottableDirectories=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEncryptionZones=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReconstructionBlocks=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptBlocks=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastCheckpointTime=1697385856136 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ReadLockLongHoldCount=0 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumActiveClients=1 2023-10-15T16:14:17,490 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastLogRoll=431 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumTimedOutPendingReconstructions=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:VolumeFailuresTotal=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ProvidedCapacityTotal=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingECBlockGroups=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedGB=0.0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecommissioningDataNodes=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.HAState=active 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyECBlockGroups=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:FilesTotal=109 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ScheduledReplicationBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureECBlockGroups=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptECBlockGroups=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MillisSinceLastLoadedEdits=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalECBlockGroups=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExcessBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalLoad=9 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExpiredHeartbeats=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInServiceLiveDataNodes=3 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumStaleStorages=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:WriteLockLongHoldCount=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlocksTotal=42 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:UnderReplicatedBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicatedBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsed=1391792 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalReplicatedBlocks=42 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:StaleDataNodes=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumFilesUnderConstruction=1 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemaining=2823754896354 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PostponedMisreplicatedBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedNonDFS=178447369040 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LockQueueLength=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptReplicatedBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Context=dfs 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomDeadDataNodes=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:Snapshots=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionECBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.TotalSyncTimes=11 5 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReplicationBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotal=3163767177216 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDeadDataNodes=0 2023-10-15T16:14:17,491 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplOneBlocks=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalSyncCount=388 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastCheckpoint=431 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicationOneBlocks=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumLiveDataNodes=3 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlockCapacity=4194304 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotalGB=2946.0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyBlocks=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomLiveDataNodes=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemainingGB=2630.0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:ReceivedBytes=15456 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeAvgTime=0.0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcSlowCalls=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationSuccesses=46 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumDroppedConnections=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcClientBackoff=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationFailures=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumOpenConnections=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.port=35085 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:SentBytes=9822 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeNumOps=46 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationSuccesses=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:CallQueueLength=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Context=rpc 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationFailures=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeNumOps=46 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeNumOps=46 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:receivedBytes=288613 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationSuccesses=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_max=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:14:17,492 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFailures=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numOpenConnections=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_num_ops=1195 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_median=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationSuccesses=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_num_ops=1195 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_num_ops=1195 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_max=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_max=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_median=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.otherExceptions=5 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Context=regionserver 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_median=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_min=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_median=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_max=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_min=0 2023-10-15T16:14:17,493 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_min=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveHandler=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_num_ops=1195 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_mean=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReadHandler=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_median=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_num_ops=1195 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_mean=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callDropped=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions=5 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFallbacks=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationFailures=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_min=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:14:17,496 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_max=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:queueSize=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveScanHandler=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_min=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:sentBytes=44933 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_mean=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_75th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_max=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_90th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_min=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_median=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_num_ops=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOnHeapSize=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_num_ops=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_num_ops=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_median=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_num_ops=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_75th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_min=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOffHeapSize=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_25th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_75th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_mean=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_25th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_25th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_mean=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tunerDoNothingCounter=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_25th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_num_ops=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:aboveHeapOccupancyLowWaterMarkCounter=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_98th_percentile=0 2023-10-15T16:14:17,497 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreSize=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_min=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_max=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_90th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_mean=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_75th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Context=regionserver 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99.9th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_mean=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_min=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockCacheSize=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_90th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockedFlushGauge=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_max=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_median=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_98th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:unblockedFlushGauge=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_max=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_max=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_num_ops=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_min=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_95th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_90th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_95th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_max=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_median=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_98th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_min=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99.9th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_median=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_median=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_95th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_mean=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_95th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_98th_percentile=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalDataFileIos=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalFileIoErrors=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateNumOps=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateNumOps=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateNumOps=0 2023-10-15T16:14:17,498 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateNumOps=0 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateNumOps=0 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateNumOps=0 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Context=dfs 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalMetadataOperations=0 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:HttpPort=null 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XceiverCount=3 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SendPacketDownstreamAvgInfo=null 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"0","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"600"}] 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:ClusterId=testClusterID 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3":{"numBlocks":21,"usedSpace":108926,"freeSpace":470660272128,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4":{"numBlocks":21,"usedSpace":516096,"freeSpace":470526299126,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":133973002}} 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DatanodeHostname=127.0.0.1 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SecurityEnabled=false 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:14:17,499 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:RpcPort=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DataPort=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SoftwareVersion=3.2.4 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SlowDisks=null 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XmitsInProgress=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DNStartedTimeInMillis=1697385857021 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsBatchedInSync=44 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsNumOps=387 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsAvgTime=0.11764705882352941 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportNumOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeNumOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteSnapshotOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsAvgTime=0.09523809523809523 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AllowSnapshotOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:RenameSnapshotOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeAvgTime=0.08333333333333333 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesCreated=168 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesAppended=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportAvgTime=0.0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageAvgTime=0.0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeNumOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeNumOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeNumOps=126 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeAvgTime=0.0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumTimesReReplicationNotScheduled=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SafeModeTime=1 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalNumOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetListingOps=132 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.ProcessName=NameNode 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageNumOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TotalFileOps=1303 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSymlinkOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AddBlockOps=49 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DisallowSnapshotOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteFileOps=23 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsNumOps=431 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSnapshotOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PendingEditsCount=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ListSnapshottableDirOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockReceivedAndDeletedOps=159 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.SessionId=null 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditNumOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FsImageLoadTime=23 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateFileOps=60 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Context=dfs 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesRenamed=41 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TimeoutReReplications=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsQueued=1 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedNumOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageNumOps=0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageAvgTime=0.0 2023-10-15T16:14:17,500 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesInGetListingOps=146 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FileInfoOps=675 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetLinkTargetOps=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetAdditionalDatanodeOps=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsBatched=68 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesDeleted=39 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SuccessfulReReplications=21 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetBlockLocations=322 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesTruncated=1 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SnapshotDiffReportOps=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportNumOps=6 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeNumOps=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditAvgTime=0.0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportAvgTime=0.3333333333333333 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumFailedVolumes=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheUsed=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheCapacity=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksCached=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:LastVolumeFailureDate=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Capacity=1054589059072 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Context=FSDatasetState 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:DfsUsed=383910 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Remaining=941226220534 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToCache=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToUnCache=7 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateAvgTime=0.0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalDataFileIos=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateNumOps=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalFileIoErrors=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateNumOps=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateNumOps=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateAvgTime=0.0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateAvgTime=0.0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateNumOps=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateAvgTime=0.0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateNumOps=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateAvgTime=0.0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateNumOps=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateNumOps=0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateAvgTime=0.0 2023-10-15T16:14:17,501 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateNumOps=0 2023-10-15T16:14:17,513 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateNumOps=0 2023-10-15T16:14:17,513 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Context=dfs 2023-10-15T16:14:17,513 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalMetadataOperations=0 2023-10-15T16:14:17,513 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateAvgTime=0.0 2023-10-15T16:14:17,513 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateAvgTime=0.0 2023-10-15T16:14:17,514 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,514 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessNumOps=1 2023-10-15T16:14:17,514 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureNumOps=0 2023-10-15T16:14:17,514 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailures=0 2023-10-15T16:14:17,514 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsNumOps=2 2023-10-15T16:14:17,514 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailuresTotal=0 2023-10-15T16:14:17,514 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Context=ugi 2023-10-15T16:14:17,514 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsAvgTime=6.0 2023-10-15T16:14:17,514 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,514 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessAvgTime=10.0 2023-10-15T16:14:17,514 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureAvgTime=0.0 2023-10-15T16:14:17,515 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfSnapshottableDirs=0 2023-10-15T16:14:17,515 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheCapacity=0 2023-10-15T16:14:17,515 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheUsed=0 2023-10-15T16:14:17,515 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ProvidedCapacity=0 2023-10-15T16:14:17,515 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:14:17,515 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DistinctVersionCount=1 2023-10-15T16:14:17,515 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocksWithReplicationFactorOne=0 2023-10-15T16:14:17,515 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentRemaining=89.249084 2023-10-15T16:14:17,515 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:14:17,515 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentUsed=4.3991608E-5 2023-10-15T16:14:17,515 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:TotalBlocks=42 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NonDfsUsedSpace=178568938320 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Safemode= 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:UpgradeFinalized=true 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:EnteringMaintenanceNodes={} 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentBlockPoolUsed=4.3991608E-5 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:LiveNodes={"127.0.0.1:40557":{"infoAddr":"127.0.0.1:35357","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:40557","lastContact":0,"usedSpace":625022,"adminState":"In Service","nonDfsUsedSpace":59546908290,"capacity":1054589059072,"numBlocks":41,"version":"3.2.4","used":625022,"remaining":941187013622,"blockScheduled":1,"blockPoolUsed":625022,"blockPoolUsedPercent":5.9266877E-5,"volfails":0,"lastBlockReport":10},"127.0.0.1:46365":{"infoAddr":"127.0.0.1:35919","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46365","lastContact":0,"usedSpace":383910,"adminState":"In Service","nonDfsUsedSpace":59531428954,"capacity":1054589059072,"numBlocks":41,"version":"3.2.4","used":383910,"remaining":941202734070,"blockScheduled":1,"blockPoolUsed":383910,"blockPoolUsedPercent":3.6403755E-5,"volfails":0,"lastBlockReport":10},"127.0.0.1:46399":{"infoAddr":"127.0.0.1:44887","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46399","lastContact":1,"usedSpace":382860,"adminState":"In Service","nonDfsUsedSpace":59490601076,"capacity":1054589059072,"numBlocks":41,"version":"3.2.4","used":382860,"remaining":941243575286,"blockScheduled":1,"blockPoolUsed":382860,"blockPoolUsedPercent":3.6304187E-5,"volfails":0,"lastBlockReport":9}} 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CompileInfo=2022-07-12T11:58Z by ubuntu from branch-3.2.4 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolId=BP-1531809096-172.17.0.3-1697385856100 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ClusterId=testClusterID 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DeadNodes={} 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:JournalTransactionInfo={"MostRecentCheckpointTxId":"0","LastAppliedOrWrittenTxId":"431"} 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Free=2823633322978 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolUsedSpace=1391792 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Threads=288 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocks=0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirStatuses={"active":{"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":"IMAGE_AND_EDITS","/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":"IMAGE_AND_EDITS"},"failed":{}} 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:SoftwareVersion=3.2.4 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:VerifyECWithTopologyResult={"isSupported":"false","resultMessage":"The number of DataNodes (3) is less than the minimum required number of DataNodes (9) for the erasure coding policies: RS-6-3-1024k"} 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Used=1391792 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Total=3163767177216 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NodeUsage={"nodeUsage":{"min":"0.00%","median":"0.00%","max":"0.00%","stdDev":"0.00%"}} 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NNStartedTimeInMillis=1697385856352 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DecomNodes={} 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CorruptFiles=[] 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirSize={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":737,"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":737} 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameJournalStatus=[{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"},{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"}] 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Old Generation=0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemMaxM=2200.0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Young Generation=328 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTerminated=0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsWaiting=86 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.SessionId=null 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Context=jvm 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapCommittedM=192.97656 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapUsedM=188.87222 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapMaxM=-1.0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTotalExtraSleepTime=36 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Hostname=8c5d72df62de 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Old Generation=0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCount=11 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsBlocked=0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogWarn=0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapCommittedM=1000.0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsRunnable=83 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumWarnThresholdExceeded=0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogInfo=0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsNew=0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapUsedM=328.47656 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTimedWaiting=119 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapMaxM=2200.0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Young Generation=11 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogError=0 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.ProcessName=DataNode 2023-10-15T16:14:17,516 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogFatal=0 2023-10-15T16:14:17,517 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillis=328 2023-10-15T16:14:17,517 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumInfoThresholdExceeded=0 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlocksTotal=42 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TopUserOpCounts={"timestamp":"2023-10-15T16:14:17+0000","windows":[{"ops":[{"opType":"*","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"getfileinfo","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]}],"windowLenMs":60000},{"ops":[{"opType":"*","totalCount":4,"allUsers":[{"user":"jenkins","count":4}],"topUsers":[{"user":"jenkins","count":4}]},{"opType":"listStatus","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"getfileinfo","totalCount":3,"allUsers":[{"user":"jenkins","count":3}],"topUsers":[{"user":"jenkins","count":3}]}],"windowLenMs":300000},{"ops":[{"opType":"*","totalCount":1424,"allUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":300},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":300},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}]},{"opType":"datanodeReport","totalCount":14,"allUsers":[{"user":"jenkins","count":14}],"topUsers":[{"user":"jenkins","count":14}]},{"opType":"listStatus","totalCount":206,"allUsers":[{"user":"jenkins","count":66},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}],"topUsers":[{"user":"jenkins","count":66},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}]},{"opType":"safemode_get","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"delete","totalCount":33,"allUsers":[{"user":"jenkins.hfs.4","count":13},{"user":"jenkins","count":12},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}],"topUsers":[{"user":"jenkins.hfs.4","count":13},{"user":"jenkins","count":12},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}]},{"opType":"getfileinfo","totalCount":675,"allUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":136},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}],"topUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":136},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}]},{"opType":"truncate","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"rename","totalCount":41,"allUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":12},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":12},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}]},{"opType":"mkdirs","totalCount":54,"allUsers":[{"user":"jenkins","count":32},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}],"topUsers":[{"user":"jenkins","count":32},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}]},{"opType":"create","totalCount":60,"allUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":17},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":17},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}]},{"opType":"setPermission","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"setTimes","totalCount":16,"allUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}]},{"opType":"open","totalCount":322,"allUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":4}],"topUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":4}]}],"windowLenMs":1500000}]} 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:SnapshotStats={"SnapshottableDirectories":0,"Snapshots":0} 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:UnderReplicatedBlocks=0 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityUsed=1391792 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEncryptionZones=0 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityRemaining=2823633322978 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReconstructionBlocks=0 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomDeadDataNodes=0 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FSState=Operational 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:VolumeFailuresTotal=0 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ProvidedCapacityTotal=0 2023-10-15T16:14:17,518 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecommissioningDataNodes=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReplicationBlocks=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncTimes=11 5 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingDeletionBlocks=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityTotal=3163767177216 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FilesTotal=109 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ScheduledReplicationBlocks=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDeadDataNodes=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncCount=388 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FsLockQueueLength=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:MaxObjects=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalLoad=9 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleDataNodes=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumLiveDataNodes=3 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlockDeletionStartTime=1697385856352 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInServiceLiveDataNodes=3 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:LowRedundancyBlocks=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomLiveDataNodes=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleStorages=0 2023-10-15T16:14:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(109): << End DataNode metrics dump 2023-10-15T16:14:18,182 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:18,755 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:14:19,182 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:20,183 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:20,357 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 17.7730 sec 2023-10-15T16:14:20,870 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:20,870 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:20,871 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:20,872 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18279 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:14:20,872 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:20,872 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:14:20,872 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:20,873 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:20,946 WARN [HBase-Metrics2-1 {}] impl.MetricsConfig(136): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-10-15T16:14:21,183 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:21,600 INFO [ProcExecTimeout {}] assignment.AssignmentManager$DeadServerMetricRegionChore(1502): Found 3 OPEN regions on dead servers and 0 OPEN regions on unknown servers 2023-10-15T16:14:22,183 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:23,183 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:24,183 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:24,253 INFO [master/8c5d72df62de:0.Chore.1 {}] master.HMaster(2059): Not running balancer because processing dead regionserver(s): 8c5d72df62de,38435,1697385859175, 8c5d72df62de,46527,1697385859222, 8c5d72df62de,35419,1697385888915, 8c5d72df62de,45999,1697385859285 2023-10-15T16:14:24,253 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(188): Hosts are {8c5d72df62de=0} racks are {/default-rack=0} 2023-10-15T16:14:24,253 DEBUG [master/8c5d72df62de:0.Chore.2 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:24,253 DEBUG [master/8c5d72df62de:0.Chore.2 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:24,253 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(287): server 0 has 3 regions 2023-10-15T16:14:24,253 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(294): server 0 is on host 0 2023-10-15T16:14:24,253 INFO [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(305): server 0 is on rack 0 2023-10-15T16:14:24,253 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(310): Number of tables=3, number of hosts=1, number of racks=1 2023-10-15T16:14:24,254 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:24,255 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:24,255 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:14:24,255 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:24,255 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:24,256 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:24,256 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is null, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:24,256 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:24,360 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:24,360 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:24,361 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:24,361 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,361 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,361 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:24,361 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:24,361 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:24,361 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:24,362 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,362 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,362 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:24,570 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:24,570 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:24,571 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:24,571 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,571 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,571 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:24,571 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:24,571 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:24,571 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:24,572 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,572 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,572 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:24,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:24,880 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:24,881 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:24,881 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,881 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,881 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:24,881 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:24,881 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:24,881 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:24,882 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,882 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24,882 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:25,184 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:25,358 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 22.7740 sec 2023-10-15T16:14:25,390 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:25,390 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:25,390 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:25,390 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:25,390 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:25,390 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:25,391 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:25,391 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:25,391 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:25,391 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:25,391 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:25,391 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:26,184 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:26,400 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:26,401 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:26,402 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:26,406 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2152 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:14:26,406 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:26,406 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:14:26,406 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:26,407 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2152 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:14:26,407 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is null, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:26,407 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:27,184 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:28,184 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:28,420 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:28,420 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:28,421 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:28,422 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4168 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:14:28,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:28,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:14:28,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:28,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:28,430 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:28,430 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:28,430 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:28,430 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4176 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:14:28,431 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:28,431 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:28,431 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:29,184 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:30,185 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:30,358 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 27.7740 sec 2023-10-15T16:14:30,940 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:30,940 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:30,941 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:30,942 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28350 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:14:30,942 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:30,942 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:14:30,942 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:30,943 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:31,185 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:32,185 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:32,450 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:32,450 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:32,450 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:32,451 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8197 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:14:32,451 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:32,451 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:32,451 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:32,451 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:32,451 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:32,452 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:32,452 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8197 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:14:32,452 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:32,452 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:32,452 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:33,185 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:34,186 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:35,186 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:35,358 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 32.7740 sec 2023-10-15T16:14:36,186 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:37,186 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:38,186 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:39,187 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:40,187 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:40,359 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 37.7750 sec 2023-10-15T16:14:40,980 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:40,980 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:40,982 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:40,982 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38390 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:14:40,986 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:40,986 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:14:40,986 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:40,986 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:41,187 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:42,187 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:42,460 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:42,460 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:42,461 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:42,461 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18207 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:14:42,462 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:42,462 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:42,462 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:42,530 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:42,531 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:42,531 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:42,531 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18277 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:14:42,532 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:42,532 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:42,532 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:43,187 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:44,188 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:45,188 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:45,359 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 42.7750 sec 2023-10-15T16:14:46,188 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:47,188 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:48,189 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:48,755 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:14:49,189 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:50,189 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:50,360 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 47.7750 sec 2023-10-15T16:14:51,000 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:51,000 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:51,001 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:14:51,002 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48410 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:14:51,002 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:51,002 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:14:51,002 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:51,003 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:51,189 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:52,189 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:52,490 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:52,490 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:52,490 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:52,490 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28237 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:14:52,491 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:52,491 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:52,491 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:52,590 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:14:52,590 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:14:52,591 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:14:52,591 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28336 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:14:52,592 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:52,592 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:52,592 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:14:53,190 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:54,190 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:55,190 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:55,360 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 52.7760 sec 2023-10-15T16:14:56,190 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:57,192 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:58,192 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:14:59,192 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:00,192 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:00,360 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 57.7760 sec 2023-10-15T16:15:01,193 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:02,193 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:02,520 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:02,520 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:02,521 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:15:02,522 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38268 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:15:02,522 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:02,522 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:15:02,522 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:02,523 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:02,601 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncNonMetaRegionLocator(265): Failed to locate region in 'hbase:replication', row='', locateType=CURRENT org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:14:02.594Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:02.701Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:02.913Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:03.221Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:03.731Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:04.742Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:06.772Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:10.792Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:20.872Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:30.942Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:40.986Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:51.002Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:15:02,602 WARN [PEWorker-3 {}] replication.AssignReplicationQueuesProcedure(189): Failed to claim replication queues for 8c5d72df62de,46527,1697385859222, suspend 128 secs org.apache.hadoop.hbase.replication.ReplicationException: failed to listAllQueueIds, serverName=8c5d72df62de,46527,1697385859222 at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:256) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=1, exceptions: 2023-10-15T16:15:02.600Z, org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:245) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.tryScheduleRetry(AsyncRpcRetryingCaller.java:130) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.onError(AsyncRpcRetryingCaller.java:215) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:112) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:15:02,604 INFO [PEWorker-3 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=63, ppid=56, state=WAITING_TIMEOUT:ASSIGN_REPLICATION_QUEUES_CLAIM, hasLock=true; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure; timeout=128000, timestamp=1697386630604 2023-10-15T16:15:02,700 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:02,700 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:02,701 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:15:02,701 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38446 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:15:02,702 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:02,702 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:02,702 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:03,193 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:04,193 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:05,194 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:06,194 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:07,194 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:08,194 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:09,194 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:10,195 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:11,195 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:12,195 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:12,580 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:12,580 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:12,581 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:15:12,582 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48328 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:15:12,582 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:12,582 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:15:12,582 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:12,583 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:12,800 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:12,800 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:12,800 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:15:12,801 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48546 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:15:12,801 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:12,801 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:12,801 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:13,195 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:14,196 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:15,196 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:16,196 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:17,196 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:18,196 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:18,755 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:15:19,197 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:20,197 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:21,197 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:22,197 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:23,198 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:24,198 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:24,262 WARN [master/8c5d72df62de:0.Chore.1 {}] janitor.CatalogJanitor(144): Failed janitorial scan of hbase:meta table org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:14:24.256Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:24.362Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24.572Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24.882Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:25.391Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:26.407Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:28.431Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:32.452Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:42.532Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:52.592Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:02.702Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:12.801Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:493) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:461) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:415) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMetaForTableRegions(MetaTableAccessor.java:410) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.scanForReport(CatalogJanitor.java:236) ~[classes/:?] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.scan(CatalogJanitor.java:171) ~[classes/:?] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.chore(CatalogJanitor.java:136) ~[classes/:?] at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?] at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] ... 7 more Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] ... 7 more 2023-10-15T16:15:24,261 ERROR [master/8c5d72df62de:0.Chore.2 {}] normalizer.RegionNormalizerChore(50): Failed to normalize regions. org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:14:24.255Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:24.361Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24.571Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:24.881Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:25.391Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:26.406Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:28.422Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:14:32.451Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:42.462Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:14:52.491Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:02.522Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:12.582Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:493) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:461) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:422) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.fullScanTables(MetaTableAccessor.java:126) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.master.TableStateManager.getTablesInStates(TableStateManager.java:121) ~[classes/:?] at org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:2194) ~[classes/:?] at org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:48) ~[classes/:?] at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?] at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:15:24,264 INFO [master/8c5d72df62de:0.Chore.1 {}] hbase.ScheduledChore(145): Chore: 8c5d72df62de,33403,1697385859093-ClusterStatusChore missed its start time 2023-10-15T16:15:24,265 DEBUG [master/8c5d72df62de:0.Chore.2 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:24,265 DEBUG [master/8c5d72df62de:0.Chore.2 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:24,266 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:15:24,267 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:24,267 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:15:24,267 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:24,267 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:24,370 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:24,370 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:24,370 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:15:24,370 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:24,371 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:24,371 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:24,580 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:24,580 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:24,581 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:15:24,581 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:24,581 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:24,581 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:24,889 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:24,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:24,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:15:24,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:24,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:24,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:25,198 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:25,400 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:25,400 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:25,400 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:15:25,400 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:25,400 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:15:25,400 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:26,198 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:26,410 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:26,410 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:26,411 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:15:26,412 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 2146 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:15:26,412 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:26,412 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:15:26,412 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:26,413 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:27,198 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:28,199 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:28,420 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:28,420 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:28,421 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:15:28,422 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 4156 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:15:28,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:28,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:15:28,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:28,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:29,199 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:30,199 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:31,199 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:32,199 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:32,450 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:32,450 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:32,451 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:15:32,452 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 8186 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:15:32,452 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:32,452 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:15:32,452 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:32,452 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:33,200 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:34,200 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:35,200 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:36,200 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:37,201 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:38,201 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:39,201 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:40,201 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:41,202 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:42,202 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:42,490 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:42,491 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:42,492 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:15:42,492 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 18226 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:15:42,492 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:42,493 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:15:42,493 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:42,493 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:43,202 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:44,202 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:45,202 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:46,203 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:47,203 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:48,203 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:48,755 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:15:49,203 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:50,203 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:51,204 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:52,204 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:52,560 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:15:52,560 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:15:52,560 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:15:52,561 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 28295 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:15:52,561 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:52,561 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:15:52,561 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:15:52,561 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:15:53,204 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:54,204 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:55,204 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:56,205 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:57,205 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:58,205 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:15:59,205 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:00,205 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:01,206 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:02,206 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:02,580 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:16:02,580 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:16:02,581 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:16:02,582 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 38316 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:16:02,582 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:16:02,582 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:16:02,582 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:16:02,583 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:16:03,207 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:04,208 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:05,208 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:06,208 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:07,208 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:08,209 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:09,209 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:10,209 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:10,995 INFO [Block report processor {}] blockmanagement.BlockManager(2745): BLOCK* processReport 0x4fa9bb17bd40a379: from storage DS-48731714-6f21-487a-99c5-91e8e15906a5 node DatanodeRegistration(127.0.0.1:46365, datanodeUuid=5acfdfa5-d523-41fd-87ab-72d83e598754, infoPort=35919, infoSecurePort=0, ipcPort=35753, storageInfo=lv=-57;cid=testClusterID;nsid=514737428;c=1697385856100), blocks: 20, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-10-15T16:16:10,995 INFO [Block report processor {}] blockmanagement.BlockManager(2745): BLOCK* processReport 0x4fa9bb17bd40a379: from storage DS-8f5ca5e0-f4c5-41cf-a29b-1cb0179988b4 node DatanodeRegistration(127.0.0.1:46365, datanodeUuid=5acfdfa5-d523-41fd-87ab-72d83e598754, infoPort=35919, infoSecurePort=0, ipcPort=35753, storageInfo=lv=-57;cid=testClusterID;nsid=514737428;c=1697385856100), blocks: 19, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-10-15T16:16:11,209 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:12,209 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:12,630 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:16:12,630 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:16:12,631 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:16:12,632 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 48366 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:16:12,632 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:16:12,632 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:16:12,632 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:16:12,632 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:16:13,210 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:14,210 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:15,210 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:16,210 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:17,211 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:18,211 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:18,756 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:16:19,211 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:20,211 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:21,211 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:21,600 INFO [ProcExecTimeout {}] assignment.AssignmentManager$DeadServerMetricRegionChore(1502): Found 3 OPEN regions on dead servers and 0 OPEN regions on unknown servers 2023-10-15T16:16:22,212 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:23,212 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:24,212 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:25,212 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:26,212 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:27,213 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:28,213 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:29,213 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:30,213 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:31,213 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:32,214 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:32,791 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:16:32,791 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:16:32,792 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:16:32,792 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 13, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 68527 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:16:32,793 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:16:32,793 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:16:32,793 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:16:32,793 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:16:33,214 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:34,214 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:35,214 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:36,214 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:37,215 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:38,215 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:39,215 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:40,215 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:41,215 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:42,216 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:43,216 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:44,216 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:45,216 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:46,216 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:47,217 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:48,217 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:48,756 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:16:49,217 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:50,217 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:51,218 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:52,218 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:52,900 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:16:52,900 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:16:52,901 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:16:52,901 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 14, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 88636 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:16:52,902 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:16:52,902 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:16:52,902 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:16:52,902 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:16:53,218 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:54,218 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:55,218 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:56,219 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:57,219 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:58,219 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:16:59,219 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:00,219 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:01,220 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:02,220 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:03,220 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting ====> TEST TIMED OUT. PRINTING THREAD DUMP. <==== Timestamp: 2023-10-15 04:17:03,830 "Session-HouseKeeper-4d1a7915-1" prio=5 tid=556 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "regionserver/8c5d72df62de:0.procedureResultReporter" daemon prio=5 tid=485 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) "qtp651210004-550" daemon prio=5 tid=550 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "SnapshotHandlerChoreCleaner" daemon prio=5 tid=900 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "master/8c5d72df62de:0.Chore.1" daemon prio=5 tid=904 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "master:store-WAL-Roller" daemon prio=5 tid=861 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hbase.wal.AbstractWALRoller.run(AbstractWALRoller.java:180) "qtp651210004-552-acceptor-0@161485b2-ServerConnector@76c396f{HTTP/1.1, (http/1.1)}{localhost:33811}" daemon prio=3 tid=552 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "pool-248-thread-1" prio=5 tid=592 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server handler 3 on default port 44717" daemon prio=5 tid=660 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "IPC Server handler 1 on default port 35085" daemon prio=5 tid=696 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "HMaster-EventLoopGroup-3-1" daemon prio=10 tid=275 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RPCClient-NioEventLoopGroup-7-3" daemon prio=5 tid=539 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "PacketResponder: BP-1531809096-172.17.0.3-1697385856100:blk_1073741830_1006, type=LAST_IN_PIPELINE" daemon prio=5 tid=884 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.lang.Object.wait(Object.java:328) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1330) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1402) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server handler 4 on default port 35085" daemon prio=5 tid=699 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "IPC Server handler 0 on default port 39027" daemon prio=5 tid=572 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=33403" daemon prio=5 tid=768 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "IPC Server handler 1 on default port 35753" daemon prio=5 tid=611 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "qtp651210004-554" daemon prio=5 tid=554 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213" daemon prio=5 tid=746 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "qtp651210004-551" daemon prio=5 tid=551 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "pool-249-thread-1" prio=5 tid=608 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "HBase-Metrics2-1" daemon prio=5 tid=242 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "PacketResponder: BP-1531809096-172.17.0.3-1697385856100:blk_1073741830_1006, type=LAST_IN_PIPELINE" daemon prio=5 tid=885 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.lang.Object.wait(Object.java:328) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1330) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1402) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "zk-event-processor-pool-0" daemon prio=5 tid=1956 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "MiniHBaseClusterRegionServer-EventLoopGroup-12-2" daemon prio=10 tid=878 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:312) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:376) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222-EventThread" daemon prio=5 tid=2157 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) "IPC Server Responder" daemon prio=5 tid=683 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) at app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) "normalizer-worker-0" daemon prio=5 tid=899 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at app//org.apache.hadoop.hbase.master.normalizer.RegionNormalizerWorkQueue.take(RegionNormalizerWorkQueue.java:146) at app//org.apache.hadoop.hbase.master.normalizer.RegionNormalizerWorker.run(RegionNormalizerWorker.java:191) at java.base@11.0.17/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base@11.0.17/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "java.util.concurrent.ThreadPoolExecutor$Worker@652247d8[State = -1, empty queue]" daemon prio=5 tid=645 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "PEWorker-4" daemon prio=5 tid=895 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) at app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) at app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) "org.apache.hadoop.hdfs.server.namenode.FSNamesystem$LazyPersistFileScrubber@1482f030" daemon prio=5 tid=582 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.hdfs.server.namenode.FSNamesystem$LazyPersistFileScrubber.run(FSNamesystem.java:4228) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Session-HouseKeeper-19accb11-1" prio=5 tid=675 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5)" daemon prio=5 tid=713 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) "qtp651210004-553" daemon prio=5 tid=553 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Abort regionserver monitor" daemon prio=5 tid=2166 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) at java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) "master/8c5d72df62de:0:becomeActiveMaster" daemon prio=5 tid=841 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hbase.master.ActiveMasterManager.blockUntilBecomingActiveMaster(ActiveMasterManager.java:305) at app//org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2486) at app//org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:611) at app//org.apache.hadoop.hbase.master.HMaster$$Lambda$291/0x0000000800958c40.run(Unknown Source) at app//org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) at app//org.apache.hadoop.hbase.trace.TraceUtil$$Lambda$276/0x000000080075d440.run(Unknown Source) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "pool-253-thread-1" daemon prio=5 tid=706 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Abort regionserver monitor" daemon prio=5 tid=1749 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) at java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) "IPC Server handler 4 on default port 39027" daemon prio=5 tid=576 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "Common-Cleaner" daemon prio=8 tid=11 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) at java.base@11.0.17/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) at java.base@11.0.17/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:161) "pool-240-thread-1" prio=5 tid=547 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33403" daemon prio=5 tid=765 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "qtp1875028383-674" daemon prio=5 tid=674 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "DataXceiver for client DFSClient_NONMAPREDUCE_-1007166148_20 at /127.0.0.1:51178 [Receiving block BP-1531809096-172.17.0.3-1697385856100:blk_1073741830_1006]" daemon prio=5 tid=881 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) at app//org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:156) at app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) at java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) at java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) at java.base@11.0.17/java.io.DataInputStream.read(DataInputStream.java:149) at app//org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:210) at app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:971) at app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:908) at app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:173) at app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:107) at app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:292) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Parameter Sending Thread #0" daemon prio=5 tid=122 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462) at java.base@11.0.17/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361) at java.base@11.0.17/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "pool-256-thread-1" prio=5 tid=623 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server handler 2 on default port 35085" daemon prio=5 tid=697 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "IPC Server handler 2 on default port 44717" daemon prio=5 tid=659 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "NIOWorkerThread-14" daemon prio=5 tid=259 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "org.apache.hadoop.util.JvmPauseMonitor$Monitor@4fd4121a" daemon prio=5 tid=679 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "NIOWorkerThread-3" daemon prio=5 tid=247 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33403" daemon prio=5 tid=767 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "IPC Server handler 3 on default port 35085" daemon prio=5 tid=698 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "Registry-endpoints-refresh-end-points" daemon prio=5 tid=1006 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hbase.client.RegistryEndpointsRefresher.mainLoop(RegistryEndpointsRefresher.java:92) at app//org.apache.hadoop.hbase.client.RegistryEndpointsRefresher$$Lambda$698/0x0000000800bb0840.run(Unknown Source) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "nioEventLoopGroup-10-1" prio=10 tid=646 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) at app//io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:803) at app//io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457) at app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "MiniHBaseClusterRegionServer-EventLoopGroup-10-2" daemon prio=10 tid=1159 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "ReadOnlyZKClient-127.0.0.1:52753@0x365ad98b" daemon prio=5 tid=1740 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) at app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:328) at app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$813/0x0000000800c14040.run(Unknown Source) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "pool-265-thread-1" daemon prio=5 tid=724 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeEditLogRoller@6d64b734" daemon prio=5 tid=581 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeEditLogRoller.run(FSNamesystem.java:4135) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Socket Reader #1 for port 0" daemon prio=5 tid=649 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) at app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) "PEWorker-3" daemon prio=5 tid=894 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) at app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) at app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) "MiniHBaseClusterRegionServer-EventLoopGroup-5-2" daemon prio=10 tid=1093 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=38213" daemon prio=5 tid=751 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=38213" daemon prio=5 tid=754 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "NIOWorkerThread-7" daemon prio=5 tid=252 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "NIOWorkerThread-8" daemon prio=5 tid=253 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server listener on 0" daemon prio=5 tid=648 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) "DataXceiver for client DFSClient_NONMAPREDUCE_-1007166148_20 at /127.0.0.1:54142 [Receiving block BP-1531809096-172.17.0.3-1697385856100:blk_1073741830_1006]" daemon prio=5 tid=882 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) at app//org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:156) at app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) at java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) at java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) at java.base@11.0.17/java.io.DataInputStream.read(DataInputStream.java:149) at app//org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:210) at app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:971) at app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:908) at app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:173) at app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:107) at app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:292) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics" daemon prio=5 tid=364 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RPCClient-NioEventLoopGroup-7-2" daemon prio=5 tid=537 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403" daemon prio=5 tid=762 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "NIOWorkerThread-4" daemon prio=5 tid=248 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "NIOWorkerThread-1" daemon prio=5 tid=28 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Async-Client-Retry-Timer-pool-0" daemon prio=5 tid=416 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) at app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "BP-1531809096-172.17.0.3-1697385856100 heartbeating to localhost/127.0.0.1:39027" daemon prio=5 tid=655 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager.waitTillNextIBR(IncrementalBlockReportManager.java:158) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:719) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:855) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server idle connection scanner for port 35753" daemon prio=5 tid=602 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) at java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) "IPC Server handler 4 on default port 44717" daemon prio=5 tid=661 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "NIOWorkerThread-10" daemon prio=5 tid=255 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "PacketResponder: BP-1531809096-172.17.0.3-1697385856100:blk_1073741830_1006, type=LAST_IN_PIPELINE" daemon prio=5 tid=883 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.lang.Object.wait(Object.java:328) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1330) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1402) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server idle connection scanner for port 35085" daemon prio=5 tid=682 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) at java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) "qtp651210004-555" daemon prio=5 tid=555 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "NIOWorkerThread-2" daemon prio=5 tid=246 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "region-location-0" daemon prio=5 tid=531 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "surefire-forkedjvm-stream-flusher" daemon prio=5 tid=14 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213" daemon prio=5 tid=747 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "IPC Server handler 1 on default port 44717" daemon prio=5 tid=658 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "snapshot-hfile-cleaner-cache-refresher" daemon prio=5 tid=905 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) at java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) "qtp315182252-626" daemon prio=5 tid=626 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "ForkJoinPool-2-worker-13" daemon prio=5 tid=193 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628) at java.base@11.0.17/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) "MiniHBaseClusterRegionServer-EventLoopGroup-13-2" daemon prio=10 tid=1989 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Socket Reader #1 for port 0" daemon prio=5 tid=564 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) at app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) "refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2/current/BP-1531809096-172.17.0.3-1697385856100" daemon prio=5 tid=638 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "HMaster-EventLoopGroup-3-2" daemon prio=10 tid=438 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "surefire-forkedjvm-command-thread" daemon prio=5 tid=16 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232) at java.base@11.0.17/sun.nio.ch.PendingFuture.get(PendingFuture.java:178) at app//org.apache.maven.surefire.api.util.internal.Channels$2.read(Channels.java:127) at java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) at java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) at java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) at app//org.apache.maven.surefire.api.util.internal.Channels$3.readImpl(Channels.java:169) at app//org.apache.maven.surefire.api.util.internal.AbstractNoninterruptibleReadableChannel.read(AbstractNoninterruptibleReadableChannel.java:50) at app//org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:430) at app//org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:419) at app//org.apache.maven.surefire.api.stream.AbstractStreamDecoder.readMessageType(AbstractStreamDecoder.java:116) at app//org.apache.maven.surefire.booter.stream.CommandDecoder.decode(CommandDecoder.java:77) at app//org.apache.maven.surefire.booter.spi.CommandChannelDecoder.decode(CommandChannelDecoder.java:60) at app//org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:290) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5/current/BP-1531809096-172.17.0.3-1697385856100" daemon prio=5 tid=719 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "org.apache.hadoop.util.JvmPauseMonitor$Monitor@6f1e007b" daemon prio=5 tid=647 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "MiniHBaseClusterRegionServer-EventLoopGroup-11-3" daemon prio=10 tid=1871 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "weak-ref-cleaner-strictcontextstorage" daemon prio=1 tid=241 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) at java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176) at app//io.opentelemetry.context.StrictContextStorage$PendingScopes.run(StrictContextStorage.java:269) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "pool-245-thread-1" daemon prio=5 tid=642 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "qtp651210004-548" daemon prio=5 tid=548 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=38213" daemon prio=5 tid=752 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "pool-270-thread-1" prio=5 tid=689 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "NIOWorkerThread-5" daemon prio=5 tid=250 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RegionServerTracker-0" daemon prio=5 tid=928 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "MiniHBaseClusterRegionServer-EventLoopGroup-6-2" daemon prio=10 tid=404 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "pool-242-thread-1" prio=5 tid=577 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "master:store-Flusher" daemon prio=5 tid=889 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2211) at app//org.apache.hadoop.hbase.master.region.MasterRegionFlusherAndCompactor.flushLoop(MasterRegionFlusherAndCompactor.java:193) at app//org.apache.hadoop.hbase.master.region.MasterRegionFlusherAndCompactor$$Lambda$383/0x0000000800a05840.run(Unknown Source) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38213" daemon prio=5 tid=750 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "regionserver/8c5d72df62de:0.procedureResultReporter" daemon prio=5 tid=954 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) "qtp1875028383-672-acceptor-0@49fdfdbd-ServerConnector@692e91fb{HTTP/1.1, (http/1.1)}{localhost:35369}" daemon prio=3 tid=672 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3)" daemon prio=5 tid=687 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) "MiniHBaseClusterRegionServer-EventLoopGroup-4-3" daemon prio=10 tid=1090 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "BP-1531809096-172.17.0.3-1697385856100 heartbeating to localhost/127.0.0.1:39027" daemon prio=5 tid=607 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager.waitTillNextIBR(IncrementalBlockReportManager.java:158) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:719) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:855) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "M:0;8c5d72df62de:38213" daemon prio=5 tid=755 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) at app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) at app//org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:623) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "NIOWorkerThread-15" daemon prio=5 tid=260 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "java.util.concurrent.ThreadPoolExecutor$Worker@5054e7fc[State = -1, empty queue]" daemon prio=5 tid=710 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "ProcExecTimeout" daemon prio=5 tid=890 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:274) at app//org.apache.hadoop.hbase.procedure2.util.DelayedUtil.takeWithoutInterrupt(DelayedUtil.java:81) at app//org.apache.hadoop.hbase.procedure2.TimeoutExecutorThread.run(TimeoutExecutorThread.java:56) "RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175-SendThread(127.0.0.1:52753)" daemon prio=5 tid=1953 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) "MiniHBaseClusterRegionServer-EventLoopGroup-6-3" daemon prio=10 tid=405 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175-EventThread" daemon prio=5 tid=1739 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) "org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor@429a6ebe" daemon prio=5 tid=579 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor.run(LeaseManager.java:526) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Listener at localhost/35085-SendThread(127.0.0.1:52753)" daemon prio=5 tid=741 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) "RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175" daemon prio=5 tid=1747 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:341) at app//org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.replicationDone(ReplicationSourceWALReader.java:121) at app//org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:149) "MiniHBaseClusterRegionServer-EventLoopGroup-13-3" daemon prio=10 tid=2041 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "HMaster-EventLoopGroup-8-3" daemon prio=10 tid=1704 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "main" prio=5 tid=1 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/java.lang.Thread.dumpThreads(Native Method) at java.base@11.0.17/java.lang.Thread.getAllStackTraces(Thread.java:1653) at app//org.apache.hadoop.hbase.TimedOutTestsListener.buildThreadDump(TimedOutTestsListener.java:92) at app//org.apache.hadoop.hbase.TimedOutTestsListener.buildThreadDiagnosticString(TimedOutTestsListener.java:78) at app//org.apache.hadoop.hbase.TimedOutTestsListener.testFailure(TimedOutTestsListener.java:65) at app//org.junit.runner.notification.SynchronizedRunListener.testFailure(SynchronizedRunListener.java:94) at app//org.junit.runner.notification.RunNotifier$6.notifyListener(RunNotifier.java:177) at app//org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) at app//org.junit.runner.notification.RunNotifier.fireTestFailures(RunNotifier.java:173) at app//org.junit.runner.notification.RunNotifier.fireTestFailure(RunNotifier.java:167) at app//org.apache.maven.surefire.common.junit4.Notifier.fireTestFailure(Notifier.java:100) at app//org.junit.internal.runners.model.EachTestNotifier.addFailure(EachTestNotifier.java:23) at app//org.junit.internal.runners.model.EachTestNotifier.addMultipleFailureException(EachTestNotifier.java:29) at app//org.junit.internal.runners.model.EachTestNotifier.addFailure(EachTestNotifier.java:21) at app//org.junit.runners.ParentRunner.run(ParentRunner.java:419) at app//org.junit.runners.Suite.runChild(Suite.java:128) at app//org.junit.runners.Suite.runChild(Suite.java:27) at app//org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at app//org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at app//org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at app//org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at app//org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at app//org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at app//org.junit.runners.ParentRunner.run(ParentRunner.java:413) at app//org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:49) at app//org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:120) at app//org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:95) at app//org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75) at app//org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:69) at app//org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:146) at app//org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at app//org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at app//org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at app//org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) "WorkerMonitor" daemon prio=5 tid=891 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:279) at app//org.apache.hadoop.hbase.procedure2.util.DelayedUtil.takeWithoutInterrupt(DelayedUtil.java:81) at app//org.apache.hadoop.hbase.procedure2.TimeoutExecutorThread.run(TimeoutExecutorThread.java:56) "qtp1875028383-671" daemon prio=5 tid=671 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38213" daemon prio=5 tid=745 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403" daemon prio=5 tid=763 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "NIOServerCxnFactory.SelectorThread-0" daemon prio=5 tid=22 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.select(NIOServerCnxnFactory.java:426) at app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.run(NIOServerCnxnFactory.java:391) "Monitor thread for TaskMonitor" daemon prio=5 tid=362 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:325) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "PEWorker-2" daemon prio=5 tid=893 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) at app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) at app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) "MiniHBaseClusterRegionServer-EventLoopGroup-6-1" daemon prio=10 tid=337 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server idle connection scanner for port 44717" daemon prio=5 tid=650 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) at java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) "SessionTracker" daemon prio=5 tid=25 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) "RPCClient-NioEventLoopGroup-7-1" daemon prio=5 tid=536 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RPCClient-NioEventLoopGroup-7-5" daemon prio=5 tid=1008 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "MiniHBaseClusterRegionServer-EventLoopGroup-12-3" daemon prio=10 tid=879 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:312) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:376) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "8c5d72df62de:33403" daemon prio=5 tid=898 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at app//org.apache.hadoop.hbase.master.assignment.AssignmentManager.waitOnAssignQueue(AssignmentManager.java:2242) at app//org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:2264) at app//org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$600(AssignmentManager.java:111) at app//org.apache.hadoop.hbase.master.assignment.AssignmentManager$1.run(AssignmentManager.java:2204) "RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33403" daemon prio=5 tid=766 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175-SendThread(127.0.0.1:52753)" daemon prio=5 tid=1738 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) "MiniHBaseClusterRegionServer-EventLoopGroup-4-1" daemon prio=10 tid=293 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Socket Reader #1 for port 0" daemon prio=5 tid=681 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) at app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) "org.apache.hadoop.util.JvmPauseMonitor$Monitor@479fa89" daemon prio=5 tid=546 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Listener at localhost/35085-EventThread" daemon prio=5 tid=742 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) "zk-event-processor-pool-0" daemon prio=5 tid=2159 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "SyncThread:0" daemon prio=5 tid=26 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:109) "IPC Server handler 3 on default port 35753" daemon prio=5 tid=613 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "qtp1875028383-673" daemon prio=5 tid=673 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "zk-event-processor-pool-0" daemon prio=5 tid=760 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "HMaster-EventLoopGroup-1-1" daemon prio=10 tid=243 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "HMaster-EventLoopGroup-8-1" daemon prio=10 tid=740 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "HMaster-EventLoopGroup-9-1" daemon prio=10 tid=757 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=33403" daemon prio=5 tid=769 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "Reference Handler" daemon prio=10 tid=2 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/java.lang.ref.Reference.waitForReferencePendingList(Native Method) at java.base@11.0.17/java.lang.ref.Reference.processPendingReferences(Reference.java:241) at java.base@11.0.17/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213) "MiniHBaseClusterRegionServer-EventLoopGroup-13-1" daemon prio=10 tid=1966 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "org.apache.hadoop.util.JvmPauseMonitor$Monitor@3a0ed71" daemon prio=5 tid=599 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "PEWorker-1" daemon prio=5 tid=892 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) at app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) at app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) "ReadOnlyZKClient-127.0.0.1:52753@0x0072515b" daemon prio=5 tid=1955 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) at app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:328) at app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$813/0x0000000800c14040.run(Unknown Source) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RPCClient-NioEventLoopGroup-7-4" daemon prio=5 tid=1007 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "nioEventLoopGroup-12-1" prio=10 tid=678 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) at app//io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:803) at app//io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457) at app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server listener on 0" daemon prio=5 tid=563 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) "RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=33403" daemon prio=5 tid=771 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "IPC Server handler 0 on default port 35753" daemon prio=5 tid=610 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "PEWorker-5" daemon prio=5 tid=896 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) at app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) at app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) "RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175-EventThread" daemon prio=5 tid=1954 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) "IPC Server handler 0 on default port 35085" daemon prio=5 tid=693 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "NIOServerCxnFactory.SelectorThread-1" daemon prio=5 tid=23 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.select(NIOServerCnxnFactory.java:426) at app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.run(NIOServerCnxnFactory.java:391) "pool-261-thread-1" prio=5 tid=656 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38213" daemon prio=5 tid=749 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6)" daemon prio=5 tid=714 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) "regionserver/8c5d72df62de:0.procedureResultReporter" daemon prio=5 tid=1997 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) "IPC Server handler 4 on default port 35753" daemon prio=5 tid=614 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "Block report processor" daemon prio=5 tid=560 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:417) at app//org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.processQueue(BlockManager.java:5107) at app//org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.run(BlockManager.java:5096) "MiniHBaseClusterRegionServer-EventLoopGroup-11-1" daemon prio=10 tid=796 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server handler 2 on default port 35753" daemon prio=5 tid=612 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "pool-1-thread-1" daemon prio=5 tid=12 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollPort$EventHandlerTask.poll(EPollPort.java:200) at java.base@11.0.17/sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:281) at java.base@11.0.17/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "org.apache.hadoop.hdfs.PeerCache@29ffa1e2" daemon prio=5 tid=368 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) at app//org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) at app//org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "qtp651210004-549" daemon prio=5 tid=549 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "region-location-0" daemon prio=5 tid=1005 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "org.apache.hadoop.hdfs.server.blockmanagement.PendingReconstructionBlocks$PendingReconstructionMonitor@1d7dee8b" daemon prio=5 tid=569 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.hdfs.server.blockmanagement.PendingReconstructionBlocks$PendingReconstructionMonitor.run(PendingReconstructionBlocks.java:259) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Client (306965244) connection to localhost/127.0.0.1:39027 from jenkins" daemon prio=5 tid=609 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1086) at app//org.apache.hadoop.ipc.Client$Connection.run(Client.java:1133) "master/8c5d72df62de:0.Chore.2" daemon prio=5 tid=2216 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1796) at java.base@11.0.17/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128) at java.base@11.0.17/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1823) at java.base@11.0.17/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1998) at app//org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:182) at app//org.apache.hadoop.hbase.client.AdminOverAsyncAdmin.tableExists(AdminOverAsyncAdmin.java:130) at app//org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.hasData(TableReplicationQueueStorage.java:536) at app//org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.preClean(ReplicationLogCleaner.java:81) at app//org.apache.hadoop.hbase.master.cleaner.CleanerChore$$Lambda$1116/0x0000000800d55840.accept(Unknown Source) at java.base@11.0.17/java.util.ArrayList.forEach(ArrayList.java:1541) at app//org.apache.hadoop.hbase.master.cleaner.CleanerChore.preRunCleaner(CleanerChore.java:282) at app//org.apache.hadoop.hbase.master.cleaner.CleanerChore.chore(CleanerChore.java:257) at app//org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) at app//org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) at app//org.apache.hadoop.hbase.trace.TraceUtil$$Lambda$276/0x000000080075d440.run(Unknown Source) at java.base@11.0.17/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base@11.0.17/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at app//org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "master/8c5d72df62de:0:becomeActiveMaster-HFileCleaner.small.0-1697385860393" daemon prio=5 tid=907 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:546) at app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:285) at app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner$2.run(HFileCleaner.java:269) "VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4)" daemon prio=5 tid=690 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) "IPC Server handler 3 on default port 39027" daemon prio=5 tid=575 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "qtp1531759665-594-acceptor-0@7dfb85d8-ServerConnector@7aafb906{HTTP/1.1, (http/1.1)}{localhost:39221}" daemon prio=3 tid=594 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "M:1;8c5d72df62de:33403" daemon prio=5 tid=772 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) at app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) at app//org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:623) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Socket Reader #1 for port 0" daemon prio=5 tid=601 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) at app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) "FSEditLogAsync" daemon prio=5 tid=562 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:417) at app//org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.dequeueEdit(FSEditLogAsync.java:235) at app//org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.run(FSEditLogAsync.java:244) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server handler 2 on default port 39027" daemon prio=5 tid=574 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33403" daemon prio=5 tid=770 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "NIOWorkerThread-16" daemon prio=5 tid=261 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "pool-268-thread-1" prio=5 tid=670 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Listener at localhost/38557.named-queue-events-pool-0" daemon prio=5 tid=292 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at app//com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:47) at app//com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56) at app//com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:159) at app//com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Command processor" daemon prio=5 tid=606 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.processQueue(BPServiceActor.java:1291) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.run(BPServiceActor.java:1275) "Command processor" daemon prio=5 tid=686 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.processQueue(BPServiceActor.java:1291) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.run(BPServiceActor.java:1275) "RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38213" daemon prio=5 tid=748 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "ProcedureDispatcherTimeoutThread" daemon prio=5 tid=897 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) at app//org.apache.hadoop.hbase.procedure2.util.DelayedUtil.takeWithoutInterrupt(DelayedUtil.java:81) at app//org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher$TimeoutExecutorThread.run(RemoteProcedureDispatcher.java:320) "regionserver/8c5d72df62de:0.procedureResultReporter" daemon prio=5 tid=484 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) "qtp1531759665-596" daemon prio=5 tid=596 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@379268ae" daemon prio=5 tid=669 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) at java.base@11.0.17/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:114) at app//org.apache.hadoop.hdfs.net.TcpPeerServer.accept(TcpPeerServer.java:85) at app//org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:145) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1)" daemon prio=5 tid=631 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) "Timer for 'DataNode' metrics system" daemon prio=5 tid=2229 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) at java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) "regionserver/8c5d72df62de:0.procedureResultReporter" daemon prio=5 tid=957 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) "DatanodeAdminMonitor-0" daemon prio=5 tid=570 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner" daemon prio=5 tid=29 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) at java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176) at app//org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3839) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Session-HouseKeeper-4e3199e3-1" prio=5 tid=597 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Command processor" daemon prio=5 tid=654 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.processQueue(BPServiceActor.java:1291) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.run(BPServiceActor.java:1275) "refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3/current/BP-1531809096-172.17.0.3-1697385856100" daemon prio=5 tid=695 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Session-HouseKeeper-5bb3f6b3-1" prio=5 tid=629 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "java.util.concurrent.ThreadPoolExecutor$Worker@5593a5f4[State = -1, empty queue]" daemon prio=5 tid=728 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics" daemon prio=5 tid=366 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2)" daemon prio=5 tid=632 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) "Finalizer" daemon prio=8 tid=3 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) at java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176) at java.base@11.0.17/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170) "LeaseRenewer:jenkins@localhost:39027" daemon prio=5 tid=731 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:441) at app//org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$800(LeaseRenewer.java:77) at app//org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:336) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor@53f681f5" daemon prio=5 tid=580 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor.run(FSNamesystem.java:4093) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "HMaster-EventLoopGroup-8-2" daemon prio=10 tid=1009 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "master/8c5d72df62de:0:becomeActiveMaster-HFileCleaner.large.0-1697385860384" daemon prio=5 tid=906 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at app//org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) at app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:285) at app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:254) "nioEventLoopGroup-8-1" prio=10 tid=598 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) at app//io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:803) at app//io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457) at app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "qtp1531759665-593" daemon prio=5 tid=593 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "ConnnectionExpirer" daemon prio=5 tid=21 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.zookeeper.server.NIOServerCnxnFactory$ConnectionExpirerThread.run(NIOServerCnxnFactory.java:574) "qtp315182252-624" daemon prio=5 tid=624 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "SnapshotHandlerChoreCleaner" daemon prio=5 tid=429 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor@6315f161" daemon prio=5 tid=558 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor.run(HeartbeatManager.java:541) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "regionserver/8c5d72df62de:0.procedureResultReporter" daemon prio=5 tid=486 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) "IPC Server idle connection scanner for port 39027" daemon prio=5 tid=565 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) at java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) "RedundancyMonitor" daemon prio=5 tid=557 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at java.base@11.0.17/java.lang.Thread.sleep(Thread.java:334) at java.base@11.0.17/java.util.concurrent.TimeUnit.sleep(TimeUnit.java:446) at app//org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$RedundancyMonitor.run(BlockManager.java:4833) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "MiniHBaseClusterRegionServer-EventLoopGroup-10-3" daemon prio=10 tid=1175 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server Responder" daemon prio=5 tid=603 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) at app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) "HMaster-EventLoopGroup-9-3" daemon prio=10 tid=923 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Listener at localhost/35085-SendThread(127.0.0.1:52753)" daemon prio=5 tid=758 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) "org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@44ecbf13" daemon prio=5 tid=591 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) at java.base@11.0.17/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:114) at app//org.apache.hadoop.hdfs.net.TcpPeerServer.accept(TcpPeerServer.java:85) at app//org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:145) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RPCClient-NioEventLoopGroup-7-6" daemon prio=5 tid=1010 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "ReadOnlyZKClient-127.0.0.1:52753@0x0a7feabd" daemon prio=5 tid=2158 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) at app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:328) at app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$813/0x0000000800c14040.run(Unknown Source) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "HMaster-EventLoopGroup-3-3" daemon prio=10 tid=439 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "zk-event-processor-pool-0" daemon prio=5 tid=1743 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "MiniHBaseClusterRegionServer-EventLoopGroup-12-1" daemon prio=10 tid=818 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:312) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:376) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "qtp315182252-625-acceptor-0@109265bd-ServerConnector@69e73e0b{HTTP/1.1, (http/1.1)}{localhost:45307}" daemon prio=3 tid=625 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "MiniHBaseClusterRegionServer-EventLoopGroup-11-2" daemon prio=10 tid=1179 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server handler 0 on default port 44717" daemon prio=5 tid=657 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1/current/BP-1531809096-172.17.0.3-1697385856100" daemon prio=5 tid=637 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "NIOWorkerThread-6" daemon prio=5 tid=251 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server handler 1 on default port 39027" daemon prio=5 tid=573 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) at app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) at app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) "FsDatasetAsyncDiskServiceFixer" daemon prio=5 tid=729 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer.run(HBaseTestingUtil.java:596) "RPCClient-NioEventLoopGroup-7-8" daemon prio=5 tid=1034 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "ProcessThread(sid:0 cport:52753):" daemon prio=5 tid=27 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:134) "DataXceiver for client DFSClient_NONMAPREDUCE_-1007166148_20 at /127.0.0.1:41708 [Receiving block BP-1531809096-172.17.0.3-1697385856100:blk_1073741830_1006]" daemon prio=5 tid=880 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) at app//org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:156) at app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) at app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) at java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) at java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) at java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) at java.base@11.0.17/java.io.DataInputStream.read(DataInputStream.java:149) at app//org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:210) at app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) at app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) at app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) at app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:971) at app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:908) at app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:173) at app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:107) at app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:292) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server listener on 0" daemon prio=5 tid=680 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) "IPC Server Responder" daemon prio=5 tid=651 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) at app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) "RpcClient-timer-pool-0" daemon prio=5 tid=417 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) at app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "qtp1531759665-595" daemon prio=5 tid=595 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "region-location-1" daemon prio=5 tid=534 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RPCClient-NioEventLoopGroup-7-7" daemon prio=5 tid=1011 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) at app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "AsyncFSWAL-0-hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData-prefix:8c5d72df62de,33403,1697385859093" daemon prio=5 tid=886 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Signal Dispatcher" daemon prio=9 tid=4 runnable java.lang.Thread.State: RUNNABLE "pool-1-thread-2" daemon prio=5 tid=13 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:417) at java.base@11.0.17/sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:275) at java.base@11.0.17/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403" daemon prio=5 tid=764 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38213" daemon prio=5 tid=753 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) at java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) at app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) at app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) "refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6/current/BP-1531809096-172.17.0.3-1697385856100" daemon prio=5 tid=720 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@57b6801e" daemon prio=5 tid=622 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) at java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) at java.base@11.0.17/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:114) at app//org.apache.hadoop.hdfs.net.TcpPeerServer.accept(TcpPeerServer.java:85) at app//org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:145) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "MiniHBaseClusterRegionServer-EventLoopGroup-5-1" daemon prio=10 tid=315 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4/current/BP-1531809096-172.17.0.3-1697385856100" daemon prio=5 tid=702 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RegionServer-Restarter" daemon prio=5 tid=1644 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Thread.sleep(Native Method) at app//org.apache.hadoop.hbase.replication.TestSyncReplicationStandbyKillRS.waitForRSShutdownToStartAndFinish(TestSyncReplicationStandbyKillRS.java:137) at app//org.apache.hadoop.hbase.replication.TestSyncReplicationStandbyKillRS.lambda$testStandbyKillRegionServer$1(TestSyncReplicationStandbyKillRS.java:82) at app//org.apache.hadoop.hbase.replication.TestSyncReplicationStandbyKillRS$$Lambda$960/0x0000000800ca7040.run(Unknown Source) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "NIOServerCxnFactory.AcceptThread:localhost/127.0.0.1:52753" daemon prio=5 tid=24 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.zookeeper.server.NIOServerCnxnFactory$AcceptThread.select(NIOServerCnxnFactory.java:229) at app//org.apache.zookeeper.server.NIOServerCnxnFactory$AcceptThread.run(NIOServerCnxnFactory.java:205) "RegionServerTracker-0" daemon prio=5 tid=927 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "NIOWorkerThread-13" daemon prio=5 tid=258 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "BP-1531809096-172.17.0.3-1697385856100 heartbeating to localhost/127.0.0.1:39027" daemon prio=5 tid=688 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/java.lang.Object.wait(Native Method) at app//org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager.waitTillNextIBR(IncrementalBlockReportManager.java:158) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:719) at app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:855) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "HMaster-EventLoopGroup-9-2" daemon prio=10 tid=922 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "CacheReplicationMonitor(2132964546)" daemon prio=5 tid=583 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2211) at app//org.apache.hadoop.hdfs.server.blockmanagement.CacheReplicationMonitor.run(CacheReplicationMonitor.java:181) "regionserver/8c5d72df62de:0.procedureResultReporter" daemon prio=5 tid=949 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) "MiniHBaseClusterRegionServer-EventLoopGroup-10-1" daemon prio=10 tid=774 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Listener at localhost/35085-EventThread" daemon prio=5 tid=759 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) "MiniHBaseClusterRegionServer-EventLoopGroup-4-2" daemon prio=10 tid=1078 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "HMaster-EventLoopGroup-1-2" daemon prio=10 tid=538 runnable java.lang.Thread.State: RUNNABLE at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) at app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "NIOWorkerThread-9" daemon prio=5 tid=254 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222-SendThread(127.0.0.1:52753)" daemon prio=5 tid=2156 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) "Idle-Rpc-Conn-Sweeper-pool-0" daemon prio=5 tid=418 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) at java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "zk-event-processor-pool-0" daemon prio=5 tid=743 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "OldWALsCleaner-0" daemon prio=5 tid=903 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at app//org.apache.hadoop.hbase.master.cleaner.LogCleaner.deleteFile(LogCleaner.java:172) at app//org.apache.hadoop.hbase.master.cleaner.LogCleaner.lambda$createOldWalsCleaner$1(LogCleaner.java:152) at app//org.apache.hadoop.hbase.master.cleaner.LogCleaner$$Lambda$511/0x0000000800ad5040.run(Unknown Source) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server Responder" daemon prio=5 tid=566 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) at app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) at app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) "qtp315182252-627" daemon prio=5 tid=627 timed_waiting java.lang.Thread.State: TIMED_WAITING at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "Listener at localhost/35085" daemon prio=5 tid=20 blocked java.lang.Thread.State: BLOCKED at app//org.junit.runner.notification.SynchronizedRunListener.testFailure(SynchronizedRunListener.java:94) at app//org.junit.runner.notification.RunNotifier$6.notifyListener(RunNotifier.java:177) at app//org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) at app//org.junit.runner.notification.RunNotifier.fireTestFailures(RunNotifier.java:173) at app//org.junit.runner.notification.RunNotifier.fireTestFailure(RunNotifier.java:167) at app//org.apache.maven.surefire.common.junit4.Notifier.fireTestFailure(Notifier.java:100) at app//org.junit.internal.runners.model.EachTestNotifier.addFailure(EachTestNotifier.java:23) at app//org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:370) at app//org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at app//org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at app//org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at app//org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at app//org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at app//org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at app//org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at app//org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at app//org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at app//org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at app//org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.base@11.0.17/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "NIOWorkerThread-11" daemon prio=5 tid=256 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) "IPC Server listener on 0" daemon prio=5 tid=600 runnable java.lang.Thread.State: RUNNABLE at java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) at java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) at java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) at java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) at app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) "NIOWorkerThread-12" daemon prio=5 tid=257 in Object.wait() java.lang.Thread.State: WAITING (on object monitor) at java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) at java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) at java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) at java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) at java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base@11.0.17/java.lang.Thread.run(Thread.java:829) 2023-10-15T16:17:03,870 INFO [Listener at localhost/35085 {}] hbase.ResourceChecker(175): after: replication.TestSyncReplicationStandbyKillRS#testStandbyKillRegionServer Thread=287 (was 755), OpenFileDescriptor=642 (was 1192), MaxFileDescriptor=1048576 (was 1048576), SystemLoadAverage=973 (was 865) - SystemLoadAverage LEAK? -, ProcessCount=14 (was 14), AvailableMemoryMB=8597 (was 8608) 2023-10-15T16:17:03,876 DEBUG [Listener at localhost/35085 {}] ipc.RpcConnection(127): Using SIMPLE authentication for service=MasterService, sasl=false 2023-10-15T16:17:03,882 INFO [HMaster-EventLoopGroup-9-3 {}] ipc.ServerRpcConnection(478): Connection from 172.17.0.3:48302, version=4.0.0-alpha-1-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-10-15T16:17:03,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.HMaster(3923): Client=jenkins//172.17.0.3 list replication peers, regex=1 2023-10-15T16:17:03,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(3923): Client=jenkins//172.17.0.3 list replication peers, regex=1 2023-10-15T16:17:03,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.HMaster(3923): Client=jenkins//172.17.0.3 list replication peers, regex=1 2023-10-15T16:17:03,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(3923): Client=jenkins//172.17.0.3 list replication peers, regex=1 2023-10-15T16:17:03,892 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.HMaster(3878): Client=jenkins//172.17.0.3 removing replication peer, id=1 2023-10-15T16:17:03,893 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.HMaster(3878): Client=jenkins//172.17.0.3 removing replication peer, id=1 2023-10-15T16:17:03,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] procedure2.ProcedureExecutor(1032): Stored pid=68, state=RUNNABLE:PRE_PEER_MODIFICATION, hasLock=false; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure 2023-10-15T16:17:03,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] procedure2.ProcedureExecutor(1032): Stored pid=69, state=RUNNABLE:PRE_PEER_MODIFICATION, hasLock=false; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure 2023-10-15T16:17:03,902 DEBUG [PEWorker-5 {}] procedure.MasterProcedureScheduler(591): PEER '1', shared lock count=0, exclusively locked by procId=68 2023-10-15T16:17:03,902 DEBUG [PEWorker-5 {}] procedure2.ProcedureExecutor(1403): LOCK_EVENT_WAIT pid=69, state=RUNNABLE:PRE_PEER_MODIFICATION, hasLock=false; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure 2023-10-15T16:17:03,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:03,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:03,904 DEBUG [PEWorker-4 {}] zookeeper.ZKUtil(111): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Set watcher on existing znode=/cluster2/replication/peers/1/sync-rep-state 2023-10-15T16:17:03,905 DEBUG [PEWorker-4 {}] zookeeper.ZKUtil(111): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Set watcher on existing znode=/cluster2/replication/peers/1/peer-state 2023-10-15T16:17:03,905 DEBUG [PEWorker-4 {}] zookeeper.ZKUtil(111): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Set watcher on existing znode=/cluster2/replication/peers/1/new-sync-rep-state 2023-10-15T16:17:03,909 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster2/replication/peers/1/new-sync-rep-state 2023-10-15T16:17:03,909 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/cluster2/replication/peers/1 2023-10-15T16:17:03,909 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster2/replication/peers/1/peer-state 2023-10-15T16:17:03,909 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster2/replication/peers/1/sync-rep-state 2023-10-15T16:17:03,909 DEBUG [Listener at localhost/35085-EventThread {}] zookeeper.ZKWatcher(604): master:33403-0x101b6f97f510006, quorum=127.0.0.1:52753, baseZNode=/cluster2 Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/cluster2/replication/peers/1 2023-10-15T16:17:03,917 INFO [PEWorker-4 {}] replication.RemovePeerProcedure(102): There are still 1 pending assign replication queues procedures [63] when removing peer 1, sleep 1 secs 2023-10-15T16:17:03,917 INFO [PEWorker-4 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=68, state=WAITING_TIMEOUT:POST_PEER_MODIFICATION, hasLock=true; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure; timeout=1000, timestamp=1697386624917 2023-10-15T16:17:04,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:04,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:04,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:04,220 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:04,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:04,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:04,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:04,918 INFO [PEWorker-2 {}] replication.RemovePeerProcedure(102): There are still 1 pending assign replication queues procedures [63] when removing peer 1, sleep 2 secs 2023-10-15T16:17:04,918 INFO [PEWorker-2 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=68, state=WAITING_TIMEOUT:POST_PEER_MODIFICATION, hasLock=true; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure; timeout=2000, timestamp=1697386626918 2023-10-15T16:17:05,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:05,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:05,220 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:06,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:06,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:06,221 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:06,918 INFO [PEWorker-1 {}] replication.RemovePeerProcedure(102): There are still 1 pending assign replication queues procedures [63] when removing peer 1, sleep 4 secs 2023-10-15T16:17:06,918 INFO [PEWorker-1 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=68, state=WAITING_TIMEOUT:POST_PEER_MODIFICATION, hasLock=true; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure; timeout=4000, timestamp=1697386630918 2023-10-15T16:17:07,221 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:08,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:08,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:08,221 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:09,221 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:10,221 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:10,607 DEBUG [PEWorker-3 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:10,607 DEBUG [PEWorker-3 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:10,608 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:17:10,609 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:10,609 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:17:10,609 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:10,609 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:10,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:10,721 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:10,721 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:17:10,721 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:10,721 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:10,721 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:10,919 INFO [PEWorker-5 {}] replication.RemovePeerProcedure(102): There are still 1 pending assign replication queues procedures [63] when removing peer 1, sleep 8 secs 2023-10-15T16:17:10,919 INFO [PEWorker-5 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=68, state=WAITING_TIMEOUT:POST_PEER_MODIFICATION, hasLock=true; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure; timeout=8000, timestamp=1697386638919 2023-10-15T16:17:10,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:10,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:10,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:17:10,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:10,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:10,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:11,222 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:11,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:11,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:11,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:17:11,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:11,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:11,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:11,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:11,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:11,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:17:11,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:11,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:11,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:12,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:12,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:12,222 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:12,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:12,761 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:12,762 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:17:12,762 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2154 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:17:12,762 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:12,763 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:17:12,763 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:12,763 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:13,020 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:13,020 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:13,020 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:17:13,021 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 15, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 108755 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.get(ClientProtos.java:45309) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.RawAsyncTableImpl.lambda$get$3(RawAsyncTableImpl.java:230) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.ConnectionUtils.call(ConnectionUtils.java:617) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.RawAsyncTableImpl.lambda$get$5(RawAsyncTableImpl.java:229) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 21 more 2023-10-15T16:17:13,021 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:13,021 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:13,021 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:13,222 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:14,222 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:14,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:14,790 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:14,791 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:17:14,792 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4184 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:17:14,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:14,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:17:14,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:14,792 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:15,222 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:16,223 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:17,223 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:18,223 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:18,757 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:17:18,800 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:18,800 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:18,801 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:17:18,802 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8194 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:17:18,802 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:18,802 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:17:18,802 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:18,803 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:18,919 INFO [PEWorker-4 {}] replication.RemovePeerProcedure(102): There are still 1 pending assign replication queues procedures [63] when removing peer 1, sleep 16 secs 2023-10-15T16:17:18,919 INFO [PEWorker-4 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=68, state=WAITING_TIMEOUT:POST_PEER_MODIFICATION, hasLock=true; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure; timeout=16000, timestamp=1697386654919 2023-10-15T16:17:19,223 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:20,223 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:21,224 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:22,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:22,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:22,224 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:23,224 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:24,224 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:25,224 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:25,363 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 14.7590 sec 2023-10-15T16:17:26,224 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:27,225 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:28,225 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:28,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:28,811 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:28,811 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:17:28,812 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18204 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:17:28,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:28,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:17:28,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:28,813 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:29,225 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:30,225 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:30,363 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 19.7590 sec 2023-10-15T16:17:31,226 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:32,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:32,226 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:32,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:33,030 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:33,030 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:33,031 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:17:33,034 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 16, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 128768 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:17:33,034 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:33,034 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:17:33,034 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:33,034 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:33,226 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:34,226 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:34,920 INFO [PEWorker-2 {}] replication.RemovePeerProcedure(102): There are still 1 pending assign replication queues procedures [63] when removing peer 1, sleep 32 secs 2023-10-15T16:17:34,920 INFO [PEWorker-2 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=68, state=WAITING_TIMEOUT:POST_PEER_MODIFICATION, hasLock=true; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure; timeout=32000, timestamp=1697386686920 2023-10-15T16:17:35,226 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:35,363 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 24.7590 sec 2023-10-15T16:17:36,227 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:37,227 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:38,227 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:38,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:38,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:38,891 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:17:38,891 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28283 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:17:38,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:38,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:17:38,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:38,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:39,227 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:40,228 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:40,364 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 29.7600 sec 2023-10-15T16:17:41,228 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:42,228 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:42,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:42,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:43,228 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:44,228 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:45,229 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:45,364 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 34.7600 sec 2023-10-15T16:17:46,229 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:47,229 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:48,229 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:48,757 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:17:48,970 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:48,970 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:48,971 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:17:48,972 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38364 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:17:48,972 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:48,972 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:17:48,972 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:48,973 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:49,229 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:50,230 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:50,364 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 39.7600 sec 2023-10-15T16:17:51,230 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:52,230 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:52,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:17:52,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:17:53,040 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:53,040 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:53,041 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:17:53,041 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 17, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 148775 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:17:53,041 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:53,041 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:17:53,041 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:53,042 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:53,230 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:54,231 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:55,231 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:55,365 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 44.7600 sec 2023-10-15T16:17:56,231 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:57,231 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:58,231 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:17:59,060 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:17:59,060 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:17:59,061 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:17:59,062 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48454 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:17:59,062 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:59,062 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:17:59,062 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:17:59,062 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:59,232 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:00,232 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:00,365 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 49.7610 sec 2023-10-15T16:18:01,232 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:02,232 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:03,232 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:04,233 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:05,233 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:05,365 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 54.7610 sec 2023-10-15T16:18:06,233 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:06,921 INFO [PEWorker-5 {}] replication.RemovePeerProcedure(102): There are still 1 pending assign replication queues procedures [63] when removing peer 1, sleep 64 secs 2023-10-15T16:18:06,921 INFO [PEWorker-5 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=68, state=WAITING_TIMEOUT:POST_PEER_MODIFICATION, hasLock=true; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure; timeout=64000, timestamp=1697386750921 2023-10-15T16:18:07,233 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:08,233 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:09,234 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:10,234 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:10,365 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-3(pid=63), run time 59.7610 sec 2023-10-15T16:18:10,611 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncNonMetaRegionLocator(265): Failed to locate region in 'hbase:replication', row='', locateType=CURRENT org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:17:10.609Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:10.721Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:10.930Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:11.240Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:11.751Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:17:12.763Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:14.792Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:18.802Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:28.812Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:38.892Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:48.972Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:17:59.062Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:18:10,611 WARN [PEWorker-3 {}] replication.AssignReplicationQueuesProcedure(189): Failed to claim replication queues for 8c5d72df62de,46527,1697385859222, suspend 256 secs org.apache.hadoop.hbase.replication.ReplicationException: failed to listAllQueueIds, serverName=8c5d72df62de,46527,1697385859222 at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:256) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=1, exceptions: 2023-10-15T16:18:10.610Z, org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:245) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.tryScheduleRetry(AsyncRpcRetryingCaller.java:130) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.onError(AsyncRpcRetryingCaller.java:215) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:112) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:18:10,613 INFO [PEWorker-3 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=63, ppid=56, state=WAITING_TIMEOUT:ASSIGN_REPLICATION_QUEUES_CLAIM, hasLock=true; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure; timeout=256000, timestamp=1697386946613 2023-10-15T16:18:11,234 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:12,234 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:12,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:18:12,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:18:13,200 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:18:13,200 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:18:13,201 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:18:13,202 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 18, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 168936 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:18:13,205 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:18:13,205 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:18:13,205 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:18:13,205 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:18:13,234 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:14,235 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:15,235 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:16,235 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:17,235 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:18,235 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:18,757 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:18:19,236 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:20,236 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:21,236 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:21,600 INFO [ProcExecTimeout {}] assignment.AssignmentManager$DeadServerMetricRegionChore(1502): Found 3 OPEN regions on dead servers and 0 OPEN regions on unknown servers 2023-10-15T16:18:22,236 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:23,237 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:24,237 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:25,237 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:26,237 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:27,237 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:28,238 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:29,238 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:30,238 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:31,238 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:32,238 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:32,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:18:32,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:18:33,239 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:33,330 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:18:33,330 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:18:33,331 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:18:33,331 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 19, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 189066 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:18:33,332 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:18:33,332 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:18:33,332 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:18:33,332 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:18:34,239 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:35,239 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:36,239 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:37,239 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:38,240 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:39,240 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:40,240 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:41,240 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:42,241 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:43,241 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:44,241 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:45,241 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:46,241 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:47,242 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:48,242 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:48,757 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:18:49,242 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:50,242 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:51,243 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:52,243 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:52,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:18:52,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:18:53,243 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:53,500 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:18:53,500 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:18:53,501 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:18:53,501 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 20, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 209236 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:18:53,502 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:18:53,502 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:18:53,502 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:18:53,502 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:18:54,243 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:55,243 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:56,244 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:57,244 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:58,244 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:18:59,244 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:00,244 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:01,245 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:02,245 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:03,245 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:04,245 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:05,246 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:06,246 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:07,246 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:08,246 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:09,246 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:10,247 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:10,922 INFO [PEWorker-4 {}] replication.RemovePeerProcedure(102): There are still 1 pending assign replication queues procedures [63] when removing peer 1, sleep 128 secs 2023-10-15T16:19:10,922 INFO [PEWorker-4 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=68, state=WAITING_TIMEOUT:POST_PEER_MODIFICATION, hasLock=true; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure; timeout=128000, timestamp=1697386878922 2023-10-15T16:19:11,247 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:12,077 DEBUG [master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics {}] regionserver.ChunkCreator$MemStoreChunkPool$StatisticsThread(417): data stats (chunk size=2097152): current pool size=7, created chunk count=19, reused chunk count=14, reuseRatio=42.42% 2023-10-15T16:19:12,092 DEBUG [master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics {}] regionserver.ChunkCreator$MemStoreChunkPool$StatisticsThread(417): index stats (chunk size=209715): current pool size=0, created chunk count=0, reused chunk count=0, reuseRatio=0 2023-10-15T16:19:12,247 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:12,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:19:12,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:19:13,247 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:13,660 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:13,660 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:13,661 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:19:13,661 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 21, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 229395 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:19:13,661 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:13,661 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:19:13,661 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:13,661 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:14,247 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:15,248 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:16,248 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:17,248 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:18,248 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:18,758 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:19:19,248 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:19,761 INFO [master:store-Flusher {}] regionserver.HRegion(2885): Flushing 1595e783b53d99cd5eef43b6debb2682 4/4 column families, dataSize=205.12 KB heapSize=258.71 KB 2023-10-15T16:19:19,805 DEBUG [master:store-WAL-Roller {}] wal.AbstractWALRoller(201): WAL AsyncFSWAL 8c5d72df62de%2C33403%2C1697385859093:(num 1697385859675) roll period 900000 ms elapsed 2023-10-15T16:19:19,842 DEBUG [master:store-Flusher {}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/info/54bea21ba1284525afc9648ae5dd0b39 is 82, key is hbase:meta,,1/info:regioninfo/1697385890369/Put/seqid=0 2023-10-15T16:19:19,847 DEBUG [master:store-WAL-Roller {}] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(472): When create output stream for /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/WALs/8c5d72df62de,33403,1697385859093/8c5d72df62de%2C33403%2C1697385859093.1697386759806, exclude list is [], retry=0 2023-10-15T16:19:19,851 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-1 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK] 2023-10-15T16:19:19,851 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-2 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK] 2023-10-15T16:19:19,852 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(810): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK] 2023-10-15T16:19:19,889 INFO [master:store-Flusher {}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=1.48 KB at sequenceid=588 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/info/54bea21ba1284525afc9648ae5dd0b39 2023-10-15T16:19:19,902 INFO [master:store-WAL-Roller {}] wal.AbstractFSWAL(976): Rolled WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/WALs/8c5d72df62de,33403,1697385859093/8c5d72df62de%2C33403%2C1697385859093.1697385859675 with entries=586, filesize=242.97 KB; new WAL /user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/WALs/8c5d72df62de,33403,1697385859093/8c5d72df62de%2C33403%2C1697385859093.1697386759806 2023-10-15T16:19:19,919 DEBUG [master:store-WAL-Roller {}] wal.AbstractFSWAL(1098): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46365,DS-48731714-6f21-487a-99c5-91e8e15906a5,DISK], DatanodeInfoWithStorage[127.0.0.1:46399,DS-ed9f94ea-0327-4f14-b978-8225e24cfa96,DISK], DatanodeInfoWithStorage[127.0.0.1:40557,DS-1b0ba2c1-87d8-48a3-a8c1-8424213187dc,DISK]] 2023-10-15T16:19:19,920 DEBUG [master:store-WAL-Roller {}] wal.AbstractFSWAL(868): hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/WALs/8c5d72df62de,33403,1697385859093/8c5d72df62de%2C33403%2C1697385859093.1697385859675 is not closed yet, will try archiving it next time 2023-10-15T16:19:19,944 DEBUG [master:store-Flusher {}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/f07e9849b45040cc90fdf97b9fdb6de2 is 467, key is \x00\x00\x00\x00\x00\x00\x00D/proc:d/1697386623911/Put/seqid=0 2023-10-15T16:19:19,960 INFO [master:store-Flusher {}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=203.18 KB at sequenceid=588 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/f07e9849b45040cc90fdf97b9fdb6de2 2023-10-15T16:19:19,968 INFO [master:store-Flusher {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f07e9849b45040cc90fdf97b9fdb6de2 2023-10-15T16:19:19,992 DEBUG [master:store-Flusher {}] hfile.HFileWriterImpl(813): Len of the biggest cell in hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/rs/0dd90a85adcf4cd39f996a1e6293bb30 is 58, key is 8c5d72df62de,35419,1697385888915/rs:/1697385895471/DeleteFamily/seqid=0 2023-10-15T16:19:20,020 INFO [master:store-Flusher {}] regionserver.DefaultStoreFlusher(81): Flushed memstore data size=476 B at sequenceid=588 (bloomFilter=true), to=hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/rs/0dd90a85adcf4cd39f996a1e6293bb30 2023-10-15T16:19:20,025 INFO [master:store-Flusher {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0dd90a85adcf4cd39f996a1e6293bb30 2023-10-15T16:19:20,026 DEBUG [master:store-Flusher {}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/info/54bea21ba1284525afc9648ae5dd0b39 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/info/54bea21ba1284525afc9648ae5dd0b39 2023-10-15T16:19:20,031 INFO [master:store-Flusher {}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/info/54bea21ba1284525afc9648ae5dd0b39, entries=18, sequenceid=588, filesize=6.2 K 2023-10-15T16:19:20,032 DEBUG [master:store-Flusher {}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/f07e9849b45040cc90fdf97b9fdb6de2 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/f07e9849b45040cc90fdf97b9fdb6de2 2023-10-15T16:19:20,040 INFO [master:store-Flusher {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f07e9849b45040cc90fdf97b9fdb6de2 2023-10-15T16:19:20,040 INFO [master:store-Flusher {}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/f07e9849b45040cc90fdf97b9fdb6de2, entries=69, sequenceid=588, filesize=12.1 K 2023-10-15T16:19:20,046 DEBUG [master:store-Flusher {}] regionserver.HRegionFileSystem(500): Committing hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/rs/0dd90a85adcf4cd39f996a1e6293bb30 as hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/rs/0dd90a85adcf4cd39f996a1e6293bb30 2023-10-15T16:19:20,063 INFO [master:store-Flusher {}] regionserver.StoreFileReader(536): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0dd90a85adcf4cd39f996a1e6293bb30 2023-10-15T16:19:20,063 INFO [master:store-Flusher {}] regionserver.HStore$StoreFlusherImpl(1984): Added hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/rs/0dd90a85adcf4cd39f996a1e6293bb30, entries=4, sequenceid=588, filesize=5.3 K 2023-10-15T16:19:20,064 INFO [master:store-Flusher {}] regionserver.HRegion(3123): Finished flush of dataSize ~205.12 KB/210043, heapSize ~258.41 KB/264616, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 303ms, sequenceid=588, compaction requested=false 2023-10-15T16:19:20,064 DEBUG [master:store-Flusher {}] regionserver.HRegion(2586): Flush status journal for 1595e783b53d99cd5eef43b6debb2682: 2023-10-15T16:19:20,249 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:20,981 WARN [HBase-Metrics2-1 {}] impl.MetricsConfig(136): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-10-15T16:19:21,249 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:21,602 INFO [master/8c5d72df62de:0.Chore.1 {}] master.HMaster(2059): Not running balancer because processing dead regionserver(s): 8c5d72df62de,38435,1697385859175, 8c5d72df62de,46527,1697385859222, 8c5d72df62de,35419,1697385888915, 8c5d72df62de,45999,1697385859285 2023-10-15T16:19:21,603 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(188): Hosts are {8c5d72df62de=0} racks are {/default-rack=0} 2023-10-15T16:19:21,603 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(287): server 0 has 3 regions 2023-10-15T16:19:21,603 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(294): server 0 is on host 0 2023-10-15T16:19:21,603 INFO [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(305): server 0 is on rack 0 2023-10-15T16:19:21,603 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(310): Number of tables=3, number of hosts=1, number of racks=1 2023-10-15T16:19:21,604 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:21,604 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:21,605 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:19:21,605 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:21,605 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:19:21,605 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:21,606 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:21,716 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:21,716 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:21,716 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:19:21,716 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:21,716 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:21,716 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:21,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:21,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:21,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:19:21,921 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:21,921 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:21,921 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:22,230 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:22,230 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:22,230 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:19:22,230 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:22,230 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:22,230 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:22,249 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:22,740 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:22,740 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:22,740 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:19:22,740 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:22,740 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:22,740 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:23,249 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:23,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:23,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:23,751 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:19:23,752 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2147 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:19:23,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:23,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:19:23,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:23,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:24,249 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:25,250 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:25,780 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:25,780 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:25,781 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:19:25,781 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4177 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:19:25,782 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:25,782 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:19:25,782 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:25,782 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:26,250 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:27,250 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:28,250 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:29,250 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:29,800 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:29,800 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:29,801 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:19:29,802 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8198 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:19:29,802 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:29,802 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:19:29,802 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:29,802 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:30,251 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:31,251 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:32,251 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:32,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:19:32,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:19:33,251 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:33,800 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:33,800 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:33,801 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:19:33,801 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 22, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 249536 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:19:33,802 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:33,802 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:19:33,802 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:33,802 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:34,252 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:35,252 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:36,252 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:37,252 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:38,252 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:39,253 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:39,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:39,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:39,891 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:19:39,891 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18287 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:19:39,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:39,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:19:39,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:39,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:40,253 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:41,253 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:42,253 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:43,253 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:44,254 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:45,254 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:46,254 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:47,254 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:48,254 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:48,758 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:19:49,255 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:49,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:49,931 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:49,932 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:19:49,932 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28328 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:19:49,933 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:49,933 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:19:49,933 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:49,933 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:50,255 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:51,255 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:52,255 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:52,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:19:53,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:19:53,256 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:53,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:53,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:53,811 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:19:53,812 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 23, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 269546 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:19:53,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:53,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:19:53,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:53,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:54,256 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:55,256 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:56,256 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:57,256 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:58,257 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:59,257 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:19:59,940 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:19:59,940 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:19:59,941 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:19:59,941 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38337 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:19:59,942 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:59,942 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:19:59,942 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:19:59,942 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:00,257 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:01,257 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:02,257 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:03,258 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:04,258 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:05,258 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:06,258 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:07,258 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:08,259 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:09,259 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:10,010 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:10,010 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:10,010 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:20:10,011 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48407 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:20:10,011 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:10,011 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:20:10,011 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:10,012 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:10,259 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:11,259 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:12,259 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:13,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:20:13,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:20:13,260 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:13,900 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:13,900 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:13,901 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:20:13,902 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 24, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 289636 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:20:13,902 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:13,902 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:20:13,902 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:13,902 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:14,260 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:15,260 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:16,260 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:17,260 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:18,261 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:18,758 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:20:19,261 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:20,261 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:21,261 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:21,600 INFO [ProcExecTimeout {}] assignment.AssignmentManager$DeadServerMetricRegionChore(1502): Found 3 OPEN regions on dead servers and 0 OPEN regions on unknown servers 2023-10-15T16:20:21,611 ERROR [master/8c5d72df62de:0.Chore.1 {}] normalizer.RegionNormalizerChore(50): Failed to normalize regions. org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:19:21.605Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:21.716Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:21.921Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:22.231Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:22.740Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:19:23.752Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:25.782Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:29.802Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:39.892Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:49.933Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:19:59.942Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:10.011Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:493) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:461) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:422) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.fullScanTables(MetaTableAccessor.java:126) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.master.TableStateManager.getTablesInStates(TableStateManager.java:121) ~[classes/:?] at org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:2194) ~[classes/:?] at org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:48) ~[classes/:?] at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?] at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:20:21,613 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:21,613 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:21,614 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:20:21,615 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:21,615 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:20:21,615 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:21,615 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:21,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:21,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:21,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:20:21,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:21,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:21,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:21,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:21,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:21,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:20:21,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:21,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:21,931 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:22,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:22,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:22,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:20:22,241 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:22,241 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:22,241 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:22,261 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:22,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:22,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:22,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:20:22,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:22,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:22,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:23,262 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:23,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:23,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:23,761 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:20:23,764 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2150 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:20:23,764 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:23,764 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:20:23,764 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:23,764 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:24,262 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:25,262 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:25,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:25,771 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:25,772 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:20:25,772 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4159 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:20:25,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:25,773 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:20:25,773 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:25,773 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:26,262 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:27,262 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:28,263 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:29,263 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:29,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:29,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:29,811 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:20:29,811 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8198 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:20:29,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:29,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:20:29,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:29,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:30,263 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:31,263 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:32,264 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:33,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:20:33,264 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:33,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:20:34,090 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:34,090 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:34,091 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:20:34,091 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 25, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 309825 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:20:34,092 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:34,092 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:20:34,092 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:34,092 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:34,264 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:35,264 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:36,264 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:37,265 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:38,265 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:39,265 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:39,840 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:39,840 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:39,841 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:20:39,842 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18228 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:20:39,842 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:39,842 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:20:39,842 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:39,843 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:40,265 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:41,265 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:42,266 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:43,266 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:44,266 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:45,266 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:46,266 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:47,267 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:48,267 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:48,758 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:20:49,267 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:49,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:49,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:49,931 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:20:49,932 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28318 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:20:49,932 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:49,932 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:20:49,932 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:49,932 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:50,267 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:51,267 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:52,268 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:53,268 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:53,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:20:53,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:20:54,170 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:54,170 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:54,171 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:20:54,171 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 26, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 329905 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:20:54,171 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:54,171 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:20:54,171 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:54,172 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:54,268 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:55,268 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:56,268 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:57,269 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:58,269 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:59,269 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:20:59,960 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:20:59,960 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:20:59,961 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:20:59,962 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38348 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:20:59,970 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:59,970 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:20:59,970 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:20:59,971 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:21:00,269 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:01,269 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:02,269 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:03,270 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:04,270 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:05,270 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:06,270 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:07,270 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:08,271 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:09,271 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:10,070 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:21:10,070 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:21:10,071 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:21:10,072 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48458 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:21:10,072 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:21:10,072 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:21:10,072 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:21:10,073 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:21:10,271 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:11,271 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:12,272 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:13,272 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:13,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:21:13,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:21:14,250 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:21:14,250 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:21:14,251 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:21:14,251 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 27, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 349986 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:21:14,252 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:21:14,252 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:21:14,252 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:21:14,252 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:21:14,272 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:15,272 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:16,272 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:17,272 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:18,273 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:18,758 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:21:18,923 INFO [PEWorker-2 {}] replication.RemovePeerProcedure(102): There are still 1 pending assign replication queues procedures [63] when removing peer 1, sleep 256 secs 2023-10-15T16:21:18,923 INFO [PEWorker-2 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=68, state=WAITING_TIMEOUT:POST_PEER_MODIFICATION, hasLock=true; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure; timeout=256000, timestamp=1697387134923 2023-10-15T16:21:19,273 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:20,273 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:21,273 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:21,622 WARN [master/8c5d72df62de:0.Chore.1 {}] janitor.CatalogJanitor(144): Failed janitorial scan of hbase:meta table org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:20:21.615Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:21.720Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:21.931Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:22.241Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:22.751Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:20:23.764Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:25.773Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:29.812Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:39.842Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:49.932Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:20:59.970Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:21:10.072Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:493) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:461) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:415) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMetaForTableRegions(MetaTableAccessor.java:410) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.scanForReport(CatalogJanitor.java:236) ~[classes/:?] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.scan(CatalogJanitor.java:171) ~[classes/:?] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.chore(CatalogJanitor.java:136) ~[classes/:?] at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?] at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:21:21,624 INFO [master/8c5d72df62de:0.Chore.1 {}] hbase.ScheduledChore(145): Chore: 8c5d72df62de,33403,1697385859093-ClusterStatusChore missed its start time 2023-10-15T16:21:22,273 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:23,274 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:24,274 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:25,274 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:26,274 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:27,274 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:28,275 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:29,275 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:30,275 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:31,275 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:32,275 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:33,276 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:33,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:21:33,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:21:34,276 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:34,410 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:21:34,410 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:21:34,411 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:21:34,411 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 28, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 370146 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:21:34,412 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:21:34,412 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:21:34,412 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:21:34,412 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:21:35,276 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:36,276 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:37,276 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:38,277 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:39,277 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:40,277 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:41,277 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:42,277 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:43,278 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:44,278 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:45,278 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:46,278 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:47,278 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:48,279 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:48,759 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:21:49,279 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:50,279 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:51,279 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:52,279 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:53,280 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:53,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:21:53,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:21:54,280 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:54,420 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:21:54,420 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:21:54,421 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:21:54,421 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 29, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 390156 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:21:54,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:21:54,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:21:54,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:21:54,422 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:21:55,280 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:56,280 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:57,281 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:58,281 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:21:59,281 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:00,281 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:01,281 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:02,282 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:03,282 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:04,282 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:05,282 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:06,282 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:07,282 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:08,283 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:09,283 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:10,283 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:11,283 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:12,284 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:13,284 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:13,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:22:13,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:22:14,284 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:14,490 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:14,490 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:14,491 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:22:14,492 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 30, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 410226 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:22:14,492 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:14,492 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:22:14,492 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:14,493 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:15,284 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:16,284 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:17,285 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:18,285 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:18,759 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:22:19,285 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:20,285 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:21,285 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:21,600 INFO [ProcExecTimeout {}] assignment.AssignmentManager$DeadServerMetricRegionChore(1502): Found 3 OPEN regions on dead servers and 0 OPEN regions on unknown servers 2023-10-15T16:22:22,286 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:23,286 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:24,286 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:25,286 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:26,286 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:26,617 DEBUG [PEWorker-1 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:26,617 DEBUG [PEWorker-1 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:26,618 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:22:26,618 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:26,618 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:22:26,618 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:26,619 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:26,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:26,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:26,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:22:26,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:26,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:26,720 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:26,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:26,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:26,930 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:22:26,931 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:26,931 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:26,931 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:27,240 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:27,241 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:27,241 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:22:27,241 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:27,241 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:27,241 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:27,287 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:27,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:27,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:27,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:22:27,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:27,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:27,751 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:28,287 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:28,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:28,760 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:28,761 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:22:28,762 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2144 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:22:28,762 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:28,762 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:22:28,762 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:28,762 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:29,287 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:30,287 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:30,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:30,770 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:30,771 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:22:30,771 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4154 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:22:30,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:30,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:22:30,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:30,772 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:31,287 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:32,288 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:33,288 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:33,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:22:34,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:22:34,289 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:34,670 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:34,671 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:34,671 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:22:34,672 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 31, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 430406 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:22:34,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:34,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:22:34,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:34,672 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:34,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:34,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:34,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:22:34,810 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8193 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:22:34,811 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:34,811 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:34,811 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:35,289 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:36,289 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:37,289 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:38,289 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:39,290 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:40,290 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:40,372 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 13.7580 sec 2023-10-15T16:22:41,290 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:42,290 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:43,290 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:44,291 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:44,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:44,890 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:44,891 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:22:44,891 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18274 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:22:44,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:44,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:22:44,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:44,892 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:45,291 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:45,372 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 18.7580 sec 2023-10-15T16:22:46,291 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:47,291 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:48,291 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:48,759 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:22:49,292 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:50,292 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:50,373 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 23.7580 sec 2023-10-15T16:22:51,292 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:52,292 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:53,292 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:53,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:22:54,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:22:54,293 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:54,850 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:54,850 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:54,851 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:22:54,851 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 32, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 450585 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:22:54,852 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:54,852 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:22:54,852 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:54,852 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:54,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:22:54,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:22:54,920 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:22:54,921 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28304 ms org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at jdk.internal.reflect.GeneratedConstructorAccessor88.newInstance(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:250) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:450) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:621) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:45339) ~[hbase-protocol-shaded-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:177) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:84) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:883) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2251) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:64) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:108) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:360) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:447) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] ... 19 more 2023-10-15T16:22:54,924 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:54,925 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:54,925 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:22:55,293 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:55,373 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 28.7590 sec 2023-10-15T16:22:56,293 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:57,293 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:58,293 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:22:59,294 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:00,294 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:00,373 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 33.7590 sec 2023-10-15T16:23:01,294 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:02,294 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:03,294 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:04,295 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:05,010 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:23:05,010 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:23:05,011 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:23:05,011 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38394 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:23:05,012 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:23:05,012 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:23:05,012 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:23:05,012 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:23:05,295 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:05,373 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 38.7590 sec 2023-10-15T16:23:06,295 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:07,295 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:08,296 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:09,296 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:10,296 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:10,374 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 43.7600 sec 2023-10-15T16:23:11,296 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:12,296 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:13,297 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:13,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:23:14,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:23:14,297 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:15,039 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:23:15,040 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:23:15,040 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:23:15,041 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 33, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 470775 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:23:15,041 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:23:15,041 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:23:15,041 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:23:15,041 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication,,99999999999999' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48424 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:23:15,041 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is null, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:23:15,042 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:23:15,297 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:15,374 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 48.7600 sec 2023-10-15T16:23:16,297 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:17,297 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:18,298 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:18,759 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:23:19,298 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:20,298 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:20,374 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 53.7600 sec 2023-10-15T16:23:21,298 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:22,298 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:23,299 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:24,299 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:25,299 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:25,374 WARN [WorkerMonitor {}] procedure2.ProcedureExecutor$WorkerMonitor(2076): Worker stuck PEWorker-1(pid=63), run time 58.7600 sec 2023-10-15T16:23:26,299 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:26,621 WARN [Async-Client-Retry-Timer-pool-0 {}] client.AsyncNonMetaRegionLocator(265): Failed to locate region in 'hbase:replication', row='', locateType=CURRENT org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:22:26.618Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:26.720Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:26.931Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:27.241Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:27.751Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:28.762Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:30.772Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:34.811Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:22:44.892Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:22:54.925Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:23:05.012Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:23:15.041Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:23:26,621 WARN [PEWorker-1 {}] replication.AssignReplicationQueuesProcedure(189): Failed to claim replication queues for 8c5d72df62de,46527,1697385859222, suspend 512 secs org.apache.hadoop.hbase.replication.ReplicationException: failed to listAllQueueIds, serverName=8c5d72df62de,46527,1697385859222 at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:256) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=1, exceptions: 2023-10-15T16:23:26.620Z, org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.listAllQueueIds(TableReplicationQueueStorage.java:245) ~[hbase-replication-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.claimQueues(AssignReplicationQueuesProcedure.java:113) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:176) ~[classes/:?] at org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure.executeFromState(AssignReplicationQueuesProcedure.java:53) ~[classes/:?] at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:921) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1649) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1961) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) ~[hbase-procedure-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.tryScheduleRetry(AsyncRpcRetryingCaller.java:130) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.onError(AsyncRpcRetryingCaller.java:215) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:112) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:71) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hadoop.hbase.exceptions.TimeoutIOException: Timeout(59999ms) waiting for region location for hbase:replication, row='', replicaId=0 at org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$withTimeout$0(AsyncRegionLocator.java:87) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:23:26,622 INFO [PEWorker-1 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=63, ppid=56, state=WAITING_TIMEOUT:ASSIGN_REPLICATION_QUEUES_CLAIM, hasLock=true; org.apache.hadoop.hbase.master.replication.AssignReplicationQueuesProcedure; timeout=512000, timestamp=1697387518622 2023-10-15T16:23:27,299 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:28,299 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:29,300 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:30,300 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:31,300 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:32,300 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:33,300 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:33,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:23:34,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:23:34,301 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:35,050 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:23:35,050 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:23:35,051 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:23:35,051 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 34, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 490786 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:23:35,052 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:23:35,052 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:23:35,052 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:23:35,052 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:23:35,301 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:36,301 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:37,301 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:38,301 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:39,302 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:40,302 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:41,302 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:42,302 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:43,302 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:44,303 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:45,303 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:46,303 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:47,303 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:48,303 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:48,759 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:23:49,304 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:50,304 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:51,304 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:52,304 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:53,304 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:54,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:23:54,304 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:54,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:23:55,120 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:23:55,120 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:23:55,121 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:23:55,125 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 35, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 510859 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:23:55,125 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:23:55,125 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:23:55,125 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:23:55,125 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:23:55,305 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:56,305 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:57,305 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:58,305 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:23:59,305 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:00,306 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:01,306 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:02,306 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:03,306 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:04,306 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:05,307 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:06,307 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:07,307 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:08,307 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:09,307 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:10,308 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:11,308 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:12,077 DEBUG [master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics {}] regionserver.ChunkCreator$MemStoreChunkPool$StatisticsThread(417): data stats (chunk size=2097152): current pool size=9, created chunk count=19, reused chunk count=15, reuseRatio=44.12% 2023-10-15T16:24:12,092 DEBUG [master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics {}] regionserver.ChunkCreator$MemStoreChunkPool$StatisticsThread(417): index stats (chunk size=209715): current pool size=0, created chunk count=0, reused chunk count=0, reuseRatio=0 2023-10-15T16:24:12,308 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:13,308 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:14,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:24:14,308 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:14,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:24:15,170 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:15,170 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:15,171 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:24:15,171 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 36, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 530905 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:24:15,171 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:15,171 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:24:15,171 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:15,172 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:15,309 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:16,309 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:16,667 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(83): >> Begin NameNode metrics dump 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_num_ops=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_mean=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_num_ops=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_25th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_95th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_90th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_90th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_max=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_98th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_25th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_75th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_min=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_75th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_min=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_max=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_max=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_25th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99.9th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_75th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_mean=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99.9th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_75th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_median=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_95th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_98th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99.9th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_median=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_25th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Context=master 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_median=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_95th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_98th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_90th_percentile=0 2023-10-15T16:24:16,688 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_mean=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99.9th_percentile=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_95th_percentile=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_min=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_max=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99th_percentile=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_98th_percentile=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_mean=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_num_ops=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_min=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_median=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_num_ops=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_90th_percentile=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:HttpPort=null 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XceiverCount=3 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"2","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-15... 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:ClusterId=testClusterID 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test... 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DatanodeHostname=127.0.0.1 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SecurityEnabled=false 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:RpcPort=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DataPort=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SoftwareVersion=3.2.4 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SlowDisks=null 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XmitsInProgress=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DNStartedTimeInMillis=1697385856691 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Context=regionserver 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.port=44717 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Context=rpcdetailed 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthNumOps=50 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpNumOps=5 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWrite=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosNumOps=48 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRemoved=10 2023-10-15T16:24:16,689 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalAvgTime=0.8367346938775511 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsNumOps=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksCached=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsAvgTime=2.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DataNodeActiveXceiversCount=1 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalNumOps=400 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsNumOps=58 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesRead=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpAvgTime=50.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpAvgTime=225065.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesAvgTime=0.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionTasks=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncCount=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:NumProcessedCommands=401 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpNumOps=104 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromLocalClient=52 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:VolumeFailures=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosNumOps=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesRead=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosAvgTime=818128.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalReadTime=18 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesRead=362275 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsAvgTime=1.4 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromRemoteClient=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksUncached=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DatanodeNetworkErrors=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromLocalClient=104 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksInPendingIBR=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockVerificationFailures=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosNumOps=208 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesWrite=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosNumOps=208 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.SessionId=null 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsNumOps=1 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesWritten=1008141 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRead=104 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReplicated=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Context=dfs 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksVerified=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromRemoteClient=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosAvgTime=26277.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsNumOps=400 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksWritten=52 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsAvgTime=0.7857142857142857 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalWriteTime=21 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcDecodingTimeNanos=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesNumOps=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosNumOps=1724 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpNumOps=52 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosAvgTime=5329.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesWritten=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosAvgTime=10111.5 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpAvgTime=0.0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpNumOps=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumFailedVolumes=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheUsed=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheCapacity=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksCached=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:LastVolumeFailureDate=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Capacity=1054589059072 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Context=FSDatasetState 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:DfsUsed=1039261 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Remaining=941156091301 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToCache=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToUnCache=10 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheCleared=1 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheHit=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Context=rpc 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheUpdated=0 2023-10-15T16:24:16,690 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalDataFileIos=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalFileIoErrors=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateNumOps=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateNumOps=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateNumOps=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateNumOps=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateNumOps=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateNumOps=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Context=dfs 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalMetadataOperations=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:MissingECBlockGroups=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:LowRedundancyECBlockGroups=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:BytesInFutureECBlockGroups=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:CorruptECBlockGroups=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:PendingDeletionECBlocks=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:TotalECBlockGroups=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheCleared=1 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheHit=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Context=rpc 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheUpdated=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:LastHATransitionTime=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:HostAndPort=localhost:39027 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:State=active 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowPeersReport=null 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowDisksReport=null 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:NNRole=NameNode 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:BytesWithFutureGenerationStamps=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SecurityEnabled=false 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.port=35753 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Context=rpcdetailed 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthNumOps=55 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:ReceivedBytes=18480 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcSlowCalls=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationSuccesses=55 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumDroppedConnections=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcClientBackoff=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationFailures=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumOpenConnections=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.port=35753 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:SentBytes=11751 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeNumOps=55 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationSuccesses=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:CallQueueLength=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Context=rpc 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationFailures=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeNumOps=55 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeNumOps=55 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:ReceivedBytes=16800 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcSlowCalls=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationSuccesses=50 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumDroppedConnections=0 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:16,691 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcClientBackoff=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationFailures=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumOpenConnections=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.port=44717 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:SentBytes=10673 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeNumOps=50 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationSuccesses=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:CallQueueLength=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Context=rpc 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationFailures=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeNumOps=50 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeNumOps=50 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemMaxM=2200.0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTerminated=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsWaiting=87 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.SessionId=null 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Context=jvm 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapUsedM=193.44197 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTotalExtraSleepTime=51 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Old Generation=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCount=11 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsBlocked=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogWarn=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsRunnable=84 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogInfo=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsNew=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapUsedM=562.47656 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTimedWaiting=118 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapMaxM=2200.0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Young Generation=11 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogError=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.ProcessName=NameNode 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogFatal=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillis=328 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumFailedVolumes=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheUsed=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheCapacity=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksCached=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:LastVolumeFailureDate=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Capacity=1054589059072 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Context=FSDatasetState 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:DfsUsed=1047950 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Remaining=941156091301 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToCache=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToUnCache=10 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemMaxM=2200.0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTerminated=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsWaiting=87 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.SessionId=null 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Context=jvm 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapUsedM=193.44412 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTotalExtraSleepTime=22 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Old Generation=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCount=11 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsBlocked=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogWarn=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsRunnable=84 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogInfo=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsNew=0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapUsedM=562.47656 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTimedWaiting=118 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapMaxM=2200.0 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Young Generation=11 2023-10-15T16:24:16,692 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogError=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.ProcessName=DataNode 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogFatal=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillis=328 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_num_ops=1906 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_max=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_mean=187 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:appendCount=1906 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:lowReplicaRollRequest=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99th_percentile=336 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99.9th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:sizeRollRequest=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendSize=301542 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_max=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowAppendCount=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:successfulLogRolls=25 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99.9th_percentile=337 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_TimeRangeCount_0-1=4 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_25th_percentile=37 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendSize=9557 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_90th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendCount=1011 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_25th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_10-100=2 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_98th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_75th_percentile=297 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_75th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_max=338 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_95th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_mean=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendCount=48 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_100-1000=2 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:errorRollRequest=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:rollRequest=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_num_ops=1819 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_median=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_98th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_min=36 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_min=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_median=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Context=regionserver 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_0-1=4 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendCount=45 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_95th_percentile=329 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendSize=74104 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_median=38 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_98th_percentile=334 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_25th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_75th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_mean=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99.9th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendSize=14494 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:writtenBytes=399697 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_90th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_95th_percentile=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowSyncRollRequest=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_num_ops=1906 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_90th_percentile=321 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_min=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendCount=802 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicationOneBlocks=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:CorruptReplicatedBlocks=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicatedBlocks=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:TotalReplicatedBlocks=43 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalDataFileIos=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalFileIoErrors=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateNumOps=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateNumOps=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateNumOps=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateNumOps=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateNumOps=0 2023-10-15T16:24:16,693 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateNumOps=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Context=dfs 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalMetadataOperations=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotNumOps=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishNumOps=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotAvgTime=0.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Context=metricssystem 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSources=67 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:DroppedPubAll=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishAvgTime=0.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSources=67 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSinks=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSinks=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheCleared=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheHit=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Context=rpc 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheUpdated=149 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWrite=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosNumOps=28 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRemoved=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsNumOps=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksCached=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsAvgTime=3.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalNumOps=9 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesRead=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpAvgTime=9.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpAvgTime=6810.7692307692305 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesAvgTime=0.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionTasks=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncCount=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:NumProcessedCommands=10 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpNumOps=31 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromLocalClient=28 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:VolumeFailures=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosNumOps=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesRead=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosAvgTime=554484.75 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalReadTime=7 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesRead=57958 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsAvgTime=1.5714285714285714 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromRemoteClient=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksUncached=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DatanodeNetworkErrors=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromLocalClient=31 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksInPendingIBR=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockVerificationFailures=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosNumOps=62 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesWrite=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosNumOps=62 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.SessionId=null 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsNumOps=1 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesWritten=620489 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRead=31 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReplicated=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Context=dfs 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksVerified=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromRemoteClient=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosAvgTime=19658.18181818182 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsNumOps=9 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksWritten=28 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsAvgTime=1.0 2023-10-15T16:24:16,694 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalWriteTime=12 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcDecodingTimeNanos=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesNumOps=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosNumOps=1285 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpNumOps=28 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosAvgTime=4301.205882352941 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesWritten=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosAvgTime=9971.454545454546 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpAvgTime=0.8181818181818182 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpNumOps=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemMaxM=2200.0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTerminated=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsWaiting=87 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.SessionId=null 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Context=jvm 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapUsedM=193.44412 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTotalExtraSleepTime=62 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Old Generation=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCount=11 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsBlocked=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogWarn=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsRunnable=84 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogInfo=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsNew=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapUsedM=562.47656 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTimedWaiting=118 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapMaxM=2200.0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Young Generation=11 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogError=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.ProcessName=DataNode 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogFatal=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillis=328 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Context=regionserver 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:numTables=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99.9th_percentile=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_90th_percentile=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_95th_percentile=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterRequests=1101 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99th_percentile=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterWriteRequests=1059 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_mean=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterReadRequests=74 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_min=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_25th_percentile=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashSubmittedCount=4 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashFailedCount=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_98th_percentile=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Context=master 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_num_ops=2 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_75th_percentile=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_max=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_median=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:receivedBytes=6051329 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_98th_percentile=1 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationSuccesses=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:24:16,695 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_25th_percentile=16 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_max=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_98th_percentile=16 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_90th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFailures=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_98th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numOpenConnections=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_75th_percentile=16 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_num_ops=1334 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99.9th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_median=36 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_90th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationSuccesses=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99.9th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_max=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_num_ops=1334 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_num_ops=1334 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_max=16 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_max=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_median=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.otherExceptions=3 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_95th_percentile=16 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Context=master 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99th_percentile=16 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_median=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_min=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99.9th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_median=16 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_max=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_95th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_min=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_98th_percentile=36 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_min=16 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_SizeRangeCount_10-100=30 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99.9th_percentile=16 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveHandler=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_num_ops=1334 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_mean=16 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_SizeRangeCount_10-100=30 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReadHandler=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_median=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_90th_percentile=36 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_num_ops=1334 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_mean=36 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_95th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_25th_percentile=36 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callDropped=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_75th_percentile=36 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions=3 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFallbacks=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationFailures=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_min=36 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_95th_percentile=1 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_max=36 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:queueSize=0 2023-10-15T16:24:16,696 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99.9th_percentile=36 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveScanHandler=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_95th_percentile=36 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_98th_percentile=1 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_90th_percentile=16 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99th_percentile=36 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_min=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:sentBytes=18267 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumFailedVolumes=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheUsed=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheCapacity=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksCached=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:LastVolumeFailureDate=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Capacity=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Context=FSDatasetState 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:DfsUsed=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Remaining=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToCache=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToUnCache=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:HttpPort=null 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XceiverCount=3 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"0","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-15... 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:ClusterId=testClusterID 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test... 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DatanodeHostname=127.0.0.1 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SecurityEnabled=false 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:RpcPort=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DataPort=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SoftwareVersion=3.2.4 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SlowDisks=null 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XmitsInProgress=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DNStartedTimeInMillis=1697385857417 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalDataFileIos=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalFileIoErrors=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateNumOps=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateNumOps=0 2023-10-15T16:24:16,697 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateNumOps=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateNumOps=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateNumOps=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateNumOps=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Context=dfs 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalMetadataOperations=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumFailedVolumes=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheUsed=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheCapacity=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksCached=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:LastVolumeFailureDate=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Capacity=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Context=FSDatasetState 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:DfsUsed=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Remaining=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToCache=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToUnCache=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:ReceivedBytes=1072216 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcSlowCalls=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationSuccesses=47 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumDroppedConnections=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.NumOpenConnectionsPerUser={"jenkins":1} 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcClientBackoff=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationFailures=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumOpenConnections=1 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.port=39027 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:SentBytes=679332 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeNumOps=3059 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationSuccesses=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:CallQueueLength=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Context=rpc 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationFailures=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeNumOps=3059 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.serverName=ClientNamenodeProtocol 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeNumOps=3059 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.port=35085 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Context=rpcdetailed 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthNumOps=46 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemMaxM=2200.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTerminated=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsWaiting=87 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.SessionId=null 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Context=jvm 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapUsedM=193.44489 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTotalExtraSleepTime=45 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Old Generation=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCount=11 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsBlocked=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogWarn=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsRunnable=84 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogInfo=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsNew=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapUsedM=562.47656 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTimedWaiting=118 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapMaxM=2200.0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Young Generation=11 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogError=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.ProcessName=DataNode 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogFatal=0 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillis=328 2023-10-15T16:24:16,698 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsBatchedInSync=31 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsNumOps=224 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsAvgTime=0.13043478260869565 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportNumOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeNumOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteSnapshotOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsAvgTime=0.03896103896103896 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AllowSnapshotOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:RenameSnapshotOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeAvgTime=0.0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesCreated=80 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesAppended=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportAvgTime=0.0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageAvgTime=0.0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeNumOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeNumOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeNumOps=3 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeAvgTime=0.0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumTimesReReplicationNotScheduled=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SafeModeTime=3 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalNumOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetListingOps=66 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.ProcessName=NameNode 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageNumOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TotalFileOps=540 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSymlinkOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AddBlockOps=28 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DisallowSnapshotOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteFileOps=5 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsNumOps=255 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSnapshotOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PendingEditsCount=1 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ListSnapshottableDirOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockReceivedAndDeletedOps=96 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.SessionId=null 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditNumOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FsImageLoadTime=173 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateFileOps=36 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Context=dfs 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesRenamed=17 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TimeoutReReplications=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsQueued=1 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedNumOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageNumOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageAvgTime=0.0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesInGetListingOps=35 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FileInfoOps=270 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetLinkTargetOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetAdditionalDatanodeOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsBatched=9 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesDeleted=1 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SuccessfulReReplications=2 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetBlockLocations=118 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesTruncated=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SnapshotDiffReportOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportNumOps=6 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeNumOps=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditAvgTime=0.0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportAvgTime=0.5 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumFailedVolumes=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheCapacity=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheUsed=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:FailedStorageLocations=[Ljava.lang.String;@f2af38b 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksCached=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:LastVolumeFailureDate=0 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Capacity=1054589059072 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToUncache=10 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:DfsUsed=1039261 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Remaining=941155698085 2023-10-15T16:24:16,699 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToCache=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpNumOps=6 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWrite=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosNumOps=54 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRemoved=10 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalAvgTime=0.7474747474747475 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsNumOps=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksCached=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsAvgTime=2.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DataNodeActiveXceiversCount=1 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalNumOps=400 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsNumOps=55 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesRead=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpAvgTime=34.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpAvgTime=225066.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesAvgTime=0.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionTasks=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncCount=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:NumProcessedCommands=401 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpNumOps=99 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromLocalClient=52 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:VolumeFailures=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosNumOps=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesRead=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosAvgTime=924152.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalReadTime=29 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesRead=688752 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsAvgTime=2.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromRemoteClient=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksUncached=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DatanodeNetworkErrors=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromLocalClient=99 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksInPendingIBR=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockVerificationFailures=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosNumOps=200 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesWrite=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosNumOps=200 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.SessionId=null 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsNumOps=1 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesWritten=1008141 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRead=99 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReplicated=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Context=dfs 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksVerified=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromRemoteClient=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosAvgTime=45642.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsNumOps=400 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksWritten=52 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsAvgTime=0.696969696969697 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalWriteTime=22 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcDecodingTimeNanos=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesNumOps=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosNumOps=1724 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpNumOps=52 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosAvgTime=6378.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesWritten=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosAvgTime=9294.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpNumOps=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,700 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWrite=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosNumOps=30 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRemoved=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsNumOps=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksCached=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsAvgTime=95.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalNumOps=9 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesRead=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpAvgTime=6809.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesAvgTime=0.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionTasks=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncCount=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:NumProcessedCommands=10 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpNumOps=35 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromLocalClient=28 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:VolumeFailures=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosNumOps=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesRead=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosAvgTime=352052.375 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalReadTime=16 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesRead=75811 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsAvgTime=1.3846153846153846 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromRemoteClient=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksUncached=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DatanodeNetworkErrors=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromLocalClient=35 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksInPendingIBR=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockVerificationFailures=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosNumOps=70 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesWrite=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosNumOps=70 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.SessionId=null 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsNumOps=1 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesWritten=620489 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRead=35 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReplicated=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Context=dfs 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksVerified=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromRemoteClient=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosAvgTime=25843.916666666668 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsNumOps=9 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksWritten=28 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsAvgTime=1.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalWriteTime=12 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcDecodingTimeNanos=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesNumOps=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosNumOps=1285 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpNumOps=28 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosAvgTime=4740.882352941177 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesWritten=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosAvgTime=9895.041666666666 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpAvgTime=0.75 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpNumOps=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpNumOps=6 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWrite=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosNumOps=34 2023-10-15T16:24:16,701 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRemoved=10 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalAvgTime=0.8877551020408163 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsNumOps=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksCached=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsAvgTime=3.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DataNodeActiveXceiversCount=1 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalNumOps=400 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsNumOps=59 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesRead=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpAvgTime=10.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpAvgTime=225063.25 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesAvgTime=0.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionTasks=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncCount=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:NumProcessedCommands=402 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpNumOps=91 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromLocalClient=52 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:VolumeFailures=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosNumOps=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesRead=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosAvgTime=1214340.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalReadTime=12 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesRead=349472 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsAvgTime=1.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromRemoteClient=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksUncached=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DatanodeNetworkErrors=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromLocalClient=91 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksInPendingIBR=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockVerificationFailures=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosNumOps=182 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesWrite=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosNumOps=182 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.SessionId=null 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsNumOps=2 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesWritten=1008141 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRead=91 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReplicated=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Context=dfs 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksVerified=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromRemoteClient=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosAvgTime=53154.75 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsNumOps=400 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksWritten=52 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsAvgTime=0.8571428571428571 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalWriteTime=28 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcDecodingTimeNanos=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesNumOps=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosNumOps=1724 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpNumOps=52 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosAvgTime=5041.5 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesWritten=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosAvgTime=8707.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpNumOps=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDataNodeMessageCount=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastWrittenTransactionId=255 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:SnapshottableDirectories=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEncryptionZones=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReconstructionBlocks=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptBlocks=0 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastCheckpointTime=1697385845476 2023-10-15T16:24:16,702 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ReadLockLongHoldCount=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumActiveClients=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastLogRoll=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumTimedOutPendingReconstructions=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:VolumeFailuresTotal=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ProvidedCapacityTotal=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingECBlockGroups=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedGB=0.0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecommissioningDataNodes=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.HAState=active 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyECBlockGroups=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:FilesTotal=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ScheduledReplicationBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureECBlockGroups=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptECBlockGroups=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MillisSinceLastLoadedEdits=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalECBlockGroups=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExcessBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalLoad=32 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExpiredHeartbeats=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInServiceLiveDataNodes=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumStaleStorages=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:WriteLockLongHoldCount=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlocksTotal=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:UnderReplicatedBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicatedBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsed=495713 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalReplicatedBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:StaleDataNodes=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumFilesUnderConstruction=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemaining=2816043276636 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PostponedMisreplicatedBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedNonDFS=185085906847 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LockQueueLength=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptReplicatedBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Context=dfs 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomDeadDataNodes=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:Snapshots=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionECBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.TotalSyncTimes= 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReplicationBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotal=3163767177216 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDeadDataNodes=3 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplOneBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalSyncCount=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastCheckpoint=255 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicationOneBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumLiveDataNodes=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlockCapacity=4194304 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotalGB=2946.0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyBlocks=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomLiveDataNodes=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemainingGB=2623.0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Context=regionserver 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheCleared=1 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheHit=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Context=rpc 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheUpdated=65 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemMaxM=2200.0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTerminated=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsWaiting=87 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.SessionId= 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Context=jvm 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapUsedM=193.45454 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Old Generation=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCount=11 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsBlocked=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogWarn=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsRunnable=84 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogInfo=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsNew=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapUsedM=563.47656 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTimedWaiting=118 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapMaxM=2200.0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Young Generation=11 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogError=0 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.ProcessName=Master 2023-10-15T16:24:16,703 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogFatal=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillis=328 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsCount=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeTotal=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointPercentComplete=1.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageTotal=1 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsTotal=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointCount=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:ElapsedTime=681 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsPercentComplete=1.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsElapsedTime=7 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointElapsedTime=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModePercentComplete=1.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:PercentComplete=1.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointTotal=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeCount=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeElapsedTime=13 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageElapsedTime=62 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageCount=1 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImagePercentComplete=1.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumFailedVolumes=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheCapacity=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheUsed=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:FailedStorageLocations=[Ljava.lang.String;@4bcf1b42 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksCached=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:LastVolumeFailureDate=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Capacity=1054589059072 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToUncache=10 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:DfsUsed=1047950 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Remaining=941155435941 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToCache=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsCount=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeTotal=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointPercentComplete=1.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageTotal=1 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsTotal=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointCount=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:ElapsedTime=681 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsPercentComplete=1.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsElapsedTime=7 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointElapsedTime=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModePercentComplete=1.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:PercentComplete=1.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointTotal=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeCount=0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeElapsedTime=13 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageElapsedTime=62 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageCount=1 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImagePercentComplete=1.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RackLocalityCostFunction=0.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionCountSkewCostFunction=0.0 2023-10-15T16:24:16,704 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaHostCostFunction=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ServerLocalityCostFunction=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MemStoreSizeCostFunction=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_95th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_Overall=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ReadRequestCostFunction=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_25th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_98th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_min=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Context=master 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_WriteRequestCostFunction=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaRackCostFunction=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_PrimaryRegionCountSkewCostFunction=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:miscInvocationCount=18 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_median=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_num_ops=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_75th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MoveCostFunction=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_max=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_mean=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_CPRequestCostFunction=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_TableSkewCostFunction=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_StoreFileCostFunction=0.0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_90th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.isBalancerActive=true 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99.9th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_95th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99.9th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_median=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_mean=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_25th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_max=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_95th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_num_ops=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_25th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_98th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_min=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_mean=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_90th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_75th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_90th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_mean=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_num_ops=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_95th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_min=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_median=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_num_ops=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_mean=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_98th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_75th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_median=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99.9th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Context=regionserver 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_max=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_num_ops=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99th_percentile=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_num_ops=0 2023-10-15T16:24:16,705 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_min=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_90th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_mean=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_95th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_max=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_75th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_75th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_min=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_num_ops=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_75th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_max=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_min=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_98th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_90th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_max=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_98th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_90th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_75th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_95th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_25th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_max=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_max=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_98th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_num_ops=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99.9th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_min=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_98th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_95th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_min=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_median=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:BlockBytesScannedCount=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99.9th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_25th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_median=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_90th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_95th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_num_ops=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_max=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_mean=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_max=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_min=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_mean=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_95th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99.9th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_25th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99.9th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_90th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_mean=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_95th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_min=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_75th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_max=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_max=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_90th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_max=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_98th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_25th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_median=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_median=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_75th_percentile=0 2023-10-15T16:24:16,706 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_mean=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99.9th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_median=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_90th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_mean=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_mean=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_75th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_num_ops=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99.9th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_median=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_75th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_median=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_median=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_98th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_95th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_75th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_95th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_num_ops=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_max=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_98th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_25th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_95th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99.9th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_num_ops=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_25th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99.9th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_median=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_90th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_25th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_median=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_min=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_max=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_90th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_max=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_75th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_25th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_98th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_min=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_mean=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99.9th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_25th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_min=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_mean=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_min=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_98th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_min=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_98th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99.9th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_median=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_num_ops=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_median=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_90th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_min=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_25th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,707 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsCount=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_98th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_median=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestRatePerSecond=0.0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerMissCount=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_mean=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_median=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_98th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_95th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_median=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_90th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_5min_rate=0.0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_min=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_95th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitRatio=0.0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_75th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_90th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_num_ops=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_min=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_25th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestCount=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedOutputBytes=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_90th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkHitCount=1 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadOpsInCurThread=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_median=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestRatePerSecond=0.0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_25th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_min=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_num_ops=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_num_ops=26 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_90th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockBytesScannedCount=4389 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_max=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_75th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_max=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_mean=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_num_ops=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_25th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheEvictionCount=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99.9th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheCount=3 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:smallCompactionQueueLength=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCount=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_min=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_num_ops=15 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_mean=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowIncrementCount=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushQueueLength=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_mean=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_75th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:localBytesRead=123986 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushCount=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_median=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutateFailedCount=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_max=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorPoolAllocationBytes=1530880 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_98th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_min=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_median=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_min=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:updatesBlockedTime=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_median=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_mean=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitQueueLength=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_min=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_min=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_95th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCachingCount=3 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_25th_percentile=0 2023-10-15T16:24:16,708 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99.9th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileAge=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_median=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_75th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_max=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_95th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_num_ops=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_min=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cpRequestCount=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_90th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_median=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_max=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_num_ops=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALCount=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_98th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitRatio=0.7272727272727273 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataMissCount=1 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestCount=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissCount=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_75th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_mean=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheEvictionCount=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheFreeSize=922067336 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_95th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_median=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_75th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_max=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_median=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:activeScanners=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissCount=3 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_mean=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_max=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_max=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_95th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactionQueueLength=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_95th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_num_ops=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_max=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_1min_rate=0.0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_min=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_median=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_num_ops=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_median=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_mean=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_75th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_min=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99.9th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_median=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_25th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,709 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_25th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_90th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_num_ops=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheSize=679544 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_num_ops=6 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_num_ops=26 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_98th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_count=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_median=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowGetCount=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_median=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterRequestsCount=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_min=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_min=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_25th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_max=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileSize=836094 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_min=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileCount=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsCount=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99.9th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_98th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_num_ops=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedOutputBytes=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_max=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_num_ops=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_max=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticBloomSize=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexMissCount=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseInfoThresholdExceeded=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCount=3 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCount=3 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_max=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedToMob=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedMemstoreBytes=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheExpressHitPercent=72.72727272727273 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionServerStartTime=1697385859285 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedInputBytes=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedFromMob=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_max=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRequestCount=47 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_25th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcGetRequestCount=3 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_90th_percentile=0 2023-10-15T16:24:16,710 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCountHitPercent=72.72727272727273 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_max=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_mean=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_min=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_98th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_min=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsSize=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_mean=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowDeleteCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockedRequestCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_mean=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheAccessCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_min=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_median=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_min=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_90th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_min=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_95th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_90th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheEvictedCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_90th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_max=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:avgStoreFileAge=1819 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_25th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitRequestCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_num_ops=1 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_min=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalBytesRead=123986 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedToMob=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_max=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedInputBytes=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheSize=679544 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileCount=953 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_mean=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcScanRequestCount=26 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreOffHeapSize=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_75th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheMissCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowPutCount=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_max=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreSize=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_75th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsSize=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_mean=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_min=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_75th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_median=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_98th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_median=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Context=regionserver 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:16,711 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:scannerLeaseExpiredCount=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:minStoreFileAge=1782 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitCount=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_98th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcFullScanRequestCount=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_mean=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitSuccessCount=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_mean=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_75th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_25th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCountPrimary=3 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_max=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataBlockCount=1 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_98th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_max=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_median=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsCount=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_mean=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsCount=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCachingCount=8 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFailedInsertionCount=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_90th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsSize=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean_rate=0.0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticIndexSize=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorUsedBufferCount=2 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_min=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99.9th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissRatio=0.2727272727272727 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_75th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_max=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_mean=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_95th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMultiRequestCount=1 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheSize=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexHitCount=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_max=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_98th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedOutputBytes=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocal=0.0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_98th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_max=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_num_ops=3 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99.9th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_25th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_min=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_mean=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_15min_rate=0.0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99.9th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_min=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_95th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_75th_percentile=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_median=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_median=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_mean=0 2023-10-15T16:24:16,712 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_95th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitCount=8 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:largeCompactionQueueLength=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_mean=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutatePassedCount=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_max=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_95th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissRatio=0.0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsCount=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_min=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_mean=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_min=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:shortCircuitBytesRead=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99.9th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_median=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_min=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoMissCount=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileIndexSize=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99.9th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_mean=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerHitCount=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_98th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsSize=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCountPrimary=8 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_min=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_max=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99.9th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_90th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_90th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_num_ops=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_max=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_mean=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_max=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_min=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRowActionRequestCount=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMutateRequestCount=16 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheCount=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_25th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,713 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_mean=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataHitCount=7 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:zeroCopyBytesRead=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_98th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionCount=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_num_ops=1 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileCount=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_num_ops=1 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_median=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALSize=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_num_ops=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_mean=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_median=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheFreeSize=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:averageRegionSize=17474 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_95th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaMissCount=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_75th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:filteredReadRequestCount=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaHitCount=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_75th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_num_ops=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_95th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_median=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCount=8 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_num_ops=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.serverName=8c5d72df62de,45999,1697385859285 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReads=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowAppendCount=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:numReferenceFiles=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedFromMob=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSize=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_max=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationRatio=1.0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_90th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_median=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_mean=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_98th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadWins=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseWarnThresholdExceeded=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_max=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_25th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoHitCount=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_median=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreHeapSize=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_25th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_min=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99.9th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_25th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_num_ops=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_90th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_max=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99.9th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_max=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_median=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsSize=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_median=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:16,714 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_median=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_min=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkMissCount=2 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_95th_percentile=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFreeSize=922067336 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheHitPercent=0.0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemMaxM=2200.0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTerminated=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsWaiting=87 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.SessionId=null 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Context=jvm 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapUsedM=193.45564 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTotalExtraSleepTime=46 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Old Generation=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCount=11 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsBlocked=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogWarn=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsRunnable=84 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogInfo=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsNew=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapUsedM=564.47656 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTimedWaiting=118 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapMaxM=2200.0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Young Generation=11 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogError=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.ProcessName=NameNode 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogFatal=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillis=328 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemMaxM=2200.0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTerminated=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsWaiting=87 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.SessionId=null 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Context=jvm 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapUsedM=193.45564 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTotalExtraSleepTime=54 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Old Generation=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCount=11 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsBlocked=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogWarn=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsRunnable=85 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogInfo=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsNew=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapUsedM=564.47656 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTimedWaiting=117 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapMaxM=2200.0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Young Generation=11 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogError=0 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.ProcessName=DataNode 2023-10-15T16:24:16,715 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogFatal=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillis=328 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_90th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_25th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_num_ops=16 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99.9th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_90th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_95th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_median=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_max=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_95th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_98th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_98th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_25th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_TimeRangeCount_1-3=1 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:fsChecksumFailureCount=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_median=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_75th_percentile=2 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_min=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Context=regionserver 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_max=2 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_min=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_90th_percentile=2 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_75th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_max=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99.9th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_98th_percentile=2 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_mean=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_95th_percentile=2 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_num_ops=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_min=2 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_median=2 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99.9th_percentile=2 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_mean=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_num_ops=214 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_25th_percentile=2 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_mean=2 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_75th_percentile=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99th_percentile=2 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context=dfs 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.1=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.2=dfs 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.2=8c5d72df62de 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.1=dfs 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.1=8c5d72df62de 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.2=0 2023-10-15T16:24:16,716 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Context=master 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:numMasterWALs=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsCount=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_98th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_median=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestRatePerSecond=0.0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerMissCount=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_mean=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_median=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_98th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_95th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_median=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_90th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_5min_rate=0.0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_min=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_95th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitRatio=0.0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_75th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_90th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_num_ops=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_min=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_25th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestCount=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedOutputBytes=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_90th_percentile=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkHitCount=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadOpsInCurThread=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_median=0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestRatePerSecond=0.0 2023-10-15T16:24:16,717 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_25th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_min=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_num_ops=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_num_ops=2 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_90th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockBytesScannedCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_max=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_75th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_max=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_mean=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_num_ops=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_25th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheEvictionCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99.9th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:smallCompactionQueueLength=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_min=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_num_ops=6 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_mean=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowIncrementCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushQueueLength=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_mean=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_75th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:localBytesRead=123986 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_median=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutateFailedCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_max=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorPoolAllocationBytes=199680 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_98th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_min=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_median=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_min=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:updatesBlockedTime=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_median=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_mean=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitQueueLength=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_min=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_min=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_95th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCachingCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_25th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99.9th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileAge=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_median=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_75th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_max=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_95th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_num_ops=1 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_min=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cpRequestCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_90th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_median=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_max=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_num_ops=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_98th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitRatio=0.0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataMissCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_75th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_mean=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheEvictionCount=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheFreeSize=922070024 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_95th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_median=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_75th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_num_ops=1 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_max=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:16,718 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_median=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:activeScanners=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissCount=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_mean=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_max=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_max=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_95th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactionQueueLength=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_95th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_num_ops=1 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_max=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_1min_rate=0.0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_min=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_median=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_num_ops=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_median=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_mean=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_75th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_min=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99.9th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_median=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_25th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_25th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_90th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_num_ops=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheSize=676856 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_num_ops=2 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_98th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_count=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_median=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowGetCount=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_median=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterRequestsCount=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_min=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_min=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_25th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_max=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileSize=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_min=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileCount=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsCount=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99.9th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_98th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_num_ops=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedOutputBytes=5860 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_max=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_num_ops=0 2023-10-15T16:24:16,719 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_max=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticBloomSize=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexMissCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseInfoThresholdExceeded=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_max=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedToMob=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedMemstoreBytes=928 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheExpressHitPercent=0.0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionServerStartTime=1697385851561 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedInputBytes=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedFromMob=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_max=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRequestCount=11 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_25th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcGetRequestCount=3 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_90th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCountHitPercent=0.0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_max=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_mean=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_min=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_98th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_min=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsSize=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_mean=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowDeleteCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockedRequestCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_mean=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheAccessCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_min=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_median=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_min=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_90th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_min=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_95th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_90th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheEvictedCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_90th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_max=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:avgStoreFileAge=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_25th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitRequestCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_min=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalBytesRead=123986 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedToMob=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_max=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedInputBytes=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheSize=676856 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileCount=479 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_mean=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcScanRequestCount=2 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreOffHeapSize=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_75th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheMissCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowPutCount=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_max=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreSize=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_75th_percentile=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsSize=0 2023-10-15T16:24:16,720 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_mean=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_min=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_75th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_median=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_98th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_median=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Context=regionserver 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:scannerLeaseExpiredCount=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:minStoreFileAge=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitCount=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_98th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcFullScanRequestCount=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_mean=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitSuccessCount=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_mean=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_75th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_25th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCountPrimary=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_max=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataBlockCount=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_98th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_max=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_median=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsCount=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_mean=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsCount=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCachingCount=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFailedInsertionCount=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_90th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsSize=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean_rate=0.0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticIndexSize=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_min=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99.9th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissRatio=0.0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_75th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_max=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_mean=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_95th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMultiRequestCount=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheSize=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexHitCount=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_max=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_98th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedOutputBytes=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocal=0.0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_98th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,721 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_max=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_num_ops=3 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99.9th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_25th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_min=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_mean=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_15min_rate=0.0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99.9th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_min=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_95th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_75th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_median=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_median=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_mean=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_95th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitCount=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:largeCompactionQueueLength=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_mean=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutatePassedCount=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_max=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_95th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissRatio=0.0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsCount=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_min=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_mean=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_min=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:shortCircuitBytesRead=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99.9th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_median=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_min=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoMissCount=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileIndexSize=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99.9th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_mean=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerHitCount=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_98th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsSize=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCountPrimary=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_min=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_max=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99.9th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_90th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_90th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_num_ops=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_max=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_mean=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:16,722 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_max=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_min=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRowActionRequestCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMutateRequestCount=6 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_25th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_mean=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataHitCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:zeroCopyBytesRead=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_98th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_num_ops=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_num_ops=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_median=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALSize=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_num_ops=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_mean=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_median=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheFreeSize=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:averageRegionSize=928 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_95th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaMissCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_75th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:filteredReadRequestCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaHitCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_75th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_num_ops=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_95th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_median=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_num_ops=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.serverName=8c5d72df62de,34443,1697385851561 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReads=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowAppendCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:numReferenceFiles=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedFromMob=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSize=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_max=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_90th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_median=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_mean=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_98th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadWins=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseWarnThresholdExceeded=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_max=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_25th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoHitCount=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_median=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreHeapSize=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_25th_percentile=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_min=0 2023-10-15T16:24:16,723 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99.9th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_25th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_num_ops=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_90th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_max=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99.9th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_max=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_median=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsSize=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_median=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_median=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_min=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkMissCount=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_95th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFreeSize=922070024 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheHitPercent=0.0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsCount=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_98th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_median=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestRatePerSecond=0.0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerMissCount=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_mean=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_median=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_98th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_95th_percentile=0 2023-10-15T16:24:16,724 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_median=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_90th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_5min_rate=0.0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_min=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_95th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitRatio=0.0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_75th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_90th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_num_ops=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_min=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_25th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestCount=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedOutputBytes=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_90th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkHitCount=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadOpsInCurThread=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_median=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestRatePerSecond=0.0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_25th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_min=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_num_ops=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_num_ops=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_90th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockBytesScannedCount=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_max=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_75th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_max=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_mean=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_num_ops=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_25th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheEvictionCount=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99.9th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheCount=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:smallCompactionQueueLength=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCount=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_min=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_num_ops=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_mean=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowIncrementCount=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushQueueLength=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_mean=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_75th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:localBytesRead=123986 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushCount=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_median=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutateFailedCount=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_max=0 2023-10-15T16:24:16,725 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorPoolAllocationBytes=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_98th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_min=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_median=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_min=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:updatesBlockedTime=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_median=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_mean=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitQueueLength=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_min=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_min=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_95th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCachingCount=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_25th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99.9th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileAge=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_median=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_75th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_max=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_95th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_num_ops=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_min=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cpRequestCount=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_90th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_median=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_max=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_num_ops=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALCount=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_98th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitRatio=0.0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataMissCount=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestCount=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissCount=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_75th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_mean=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheEvictionCount=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheFreeSize=922070024 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_95th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_median=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_75th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_max=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_median=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:activeScanners=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissCount=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_mean=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_max=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_max=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_95th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactionQueueLength=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_95th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_num_ops=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_max=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_1min_rate=0.0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_min=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_median=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_num_ops=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_median=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,726 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_mean=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_75th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_min=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99.9th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_median=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_25th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_25th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_90th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_num_ops=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheSize=676856 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_num_ops=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_98th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_count=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_median=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowGetCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_median=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterRequestsCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_min=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_min=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_25th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_max=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileSize=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_min=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99.9th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_98th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_num_ops=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedOutputBytes=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_max=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_num_ops=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_max=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticBloomSize=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexMissCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseInfoThresholdExceeded=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_max=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedToMob=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedMemstoreBytes=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheExpressHitPercent=0.0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionServerStartTime=1697385851811 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedInputBytes=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedFromMob=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_max=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRequestCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_25th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcGetRequestCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_90th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCountHitPercent=0.0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_max=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_mean=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_min=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_98th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_min=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsSize=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_mean=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowDeleteCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockedRequestCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_mean=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheAccessCount=0 2023-10-15T16:24:16,727 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_min=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_median=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_min=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_90th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_min=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_95th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_90th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheEvictedCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_90th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_max=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:avgStoreFileAge=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_25th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitRequestCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_min=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalBytesRead=123986 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedToMob=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_max=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedInputBytes=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheSize=676856 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileCount=240 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_mean=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcScanRequestCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreOffHeapSize=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_75th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheMissCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowPutCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_max=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreSize=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_75th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsSize=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_mean=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_min=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_75th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_median=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_98th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_median=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Context=regionserver 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:scannerLeaseExpiredCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:minStoreFileAge=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_98th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcFullScanRequestCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_mean=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitSuccessCount=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_mean=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_75th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_25th_percentile=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCountPrimary=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_max=0 2023-10-15T16:24:16,728 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataBlockCount=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_98th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_max=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_median=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsCount=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_mean=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsCount=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCachingCount=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFailedInsertionCount=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_90th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsSize=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean_rate=0.0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticIndexSize=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorUsedBufferCount=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_min=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99.9th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissRatio=0.0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_75th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_max=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_mean=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_95th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMultiRequestCount=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheSize=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexHitCount=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_max=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_98th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedOutputBytes=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocal=0.0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_98th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_max=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_num_ops=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99.9th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_25th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_min=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_mean=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_15min_rate=0.0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99.9th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_min=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_95th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_75th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_median=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_median=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_mean=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_95th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitCount=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:largeCompactionQueueLength=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_mean=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutatePassedCount=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_max=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_95th_percentile=0 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:24:16,729 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissRatio=0.0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_min=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_mean=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_min=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:shortCircuitBytesRead=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99.9th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_median=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_min=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoMissCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileIndexSize=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99.9th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_mean=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerHitCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_98th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsSize=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCountPrimary=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_min=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_max=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99.9th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_90th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_90th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_num_ops=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_max=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_mean=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_max=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_min=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRowActionRequestCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMutateRequestCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_25th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_mean=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataHitCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:zeroCopyBytesRead=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_98th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_num_ops=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_num_ops=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_median=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALSize=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_num_ops=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_mean=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_median=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheFreeSize=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:averageRegionSize=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_95th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaMissCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_75th_percentile=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:filteredReadRequestCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaHitCount=0 2023-10-15T16:24:16,730 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_75th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_num_ops=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_95th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_median=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCount=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_num_ops=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.serverName=8c5d72df62de,40585,1697385851811 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReads=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowAppendCount=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:numReferenceFiles=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedFromMob=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSize=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_max=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_90th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_median=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_mean=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_98th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadWins=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseWarnThresholdExceeded=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_max=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_25th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoHitCount=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_median=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreHeapSize=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_25th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_min=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99.9th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_25th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_num_ops=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_90th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_max=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99.9th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_max=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_median=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsSize=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_median=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_median=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_min=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkMissCount=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_95th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFreeSize=922070024 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheHitPercent=0.0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:16,731 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsCount=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_98th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_median=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestRatePerSecond=0.0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerMissCount=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_mean=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_median=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_98th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_95th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_median=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_90th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_5min_rate=0.0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_min=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_95th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitRatio=0.0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_75th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_90th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_num_ops=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_min=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_25th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestCount=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedOutputBytes=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_90th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkHitCount=1 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadOpsInCurThread=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_median=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestRatePerSecond=0.0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_25th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_min=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_num_ops=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_num_ops=11 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_90th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockBytesScannedCount=6840 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_max=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_75th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_max=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_mean=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_num_ops=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_25th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheEvictionCount=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99.9th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheCount=4 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:smallCompactionQueueLength=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCount=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_min=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_num_ops=7 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_num_ops=1 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_mean=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowIncrementCount=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushQueueLength=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_mean=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_75th_percentile=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:localBytesRead=123986 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushCount=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_median=0 2023-10-15T16:24:16,732 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutateFailedCount=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_max=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorPoolAllocationBytes=599040 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_98th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_min=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_median=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_min=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:updatesBlockedTime=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_median=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_mean=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitQueueLength=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_min=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_min=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_95th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCachingCount=4 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_25th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99.9th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileAge=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_median=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_75th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_max=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_95th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_num_ops=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_min=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cpRequestCount=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_90th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_median=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_max=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_num_ops=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALCount=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_98th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitRatio=0.72 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataMissCount=6 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestCount=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissCount=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_75th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_mean=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheEvictionCount=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheFreeSize=922061632 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_95th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_median=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_75th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_max=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_median=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:activeScanners=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissCount=7 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_mean=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_max=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_max=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_95th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactionQueueLength=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_95th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_num_ops=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_max=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_1min_rate=0.0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_min=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_median=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_num_ops=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_median=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_mean=0 2023-10-15T16:24:16,733 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_75th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_min=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99.9th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_median=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_25th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_25th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_90th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_num_ops=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheSize=685248 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_num_ops=7 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_num_ops=11 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_98th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_count=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_median=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowGetCount=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_median=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterRequestsCount=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_min=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_min=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_25th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_num_ops=1 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_max=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileSize=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_min=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileCount=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsCount=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99.9th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_98th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_num_ops=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedOutputBytes=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_max=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_num_ops=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_max=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticBloomSize=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexMissCount=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseInfoThresholdExceeded=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCount=4 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCount=7 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_max=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedToMob=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedMemstoreBytes=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheExpressHitPercent=81.81818181818183 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionServerStartTime=1697385888915 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedInputBytes=18105 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedFromMob=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_max=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRequestCount=21 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_25th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcGetRequestCount=1 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_90th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeCount=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCountHitPercent=72.0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_max=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_mean=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_min=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_98th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_min=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsSize=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_mean=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowDeleteCount=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockedRequestCount=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_mean=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheAccessCount=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_min=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_median=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_min=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_90th_percentile=0 2023-10-15T16:24:16,734 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_min=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_95th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_90th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheEvictedCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_90th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_max=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:avgStoreFileAge=12090 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_25th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitRequestCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_num_ops=1 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_min=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalBytesRead=123986 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedToMob=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_max=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedInputBytes=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheSize=685248 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileCount=699 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_mean=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcScanRequestCount=11 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreOffHeapSize=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_75th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheMissCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowPutCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_max=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreSize=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_75th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsSize=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_mean=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_min=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_75th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_median=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_98th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_median=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Context=regionserver 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:scannerLeaseExpiredCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:minStoreFileAge=2959 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_98th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcFullScanRequestCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_mean=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitSuccessCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_mean=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_75th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_25th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCountPrimary=7 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_max=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataBlockCount=3 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_98th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_max=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_median=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_mean=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsCount=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCachingCount=18 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,735 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFailedInsertionCount=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_90th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsSize=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean_rate=0.0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticIndexSize=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_min=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99.9th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissRatio=0.28 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_75th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_max=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_mean=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_95th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMultiRequestCount=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheSize=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexHitCount=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_max=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_98th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedOutputBytes=5921 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocal=0.0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_98th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_max=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_num_ops=1 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99.9th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_25th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_min=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_mean=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_15min_rate=0.0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99.9th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_min=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_95th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_75th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_median=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_median=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_mean=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_95th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitCount=18 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:largeCompactionQueueLength=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_mean=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutatePassedCount=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_max=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_95th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissRatio=0.0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsCount=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_min=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_mean=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_min=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:shortCircuitBytesRead=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99.9th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,736 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_median=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_min=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoMissCount=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileIndexSize=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99.9th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_mean=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerHitCount=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_98th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsSize=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCountPrimary=18 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_min=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_max=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99.9th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_90th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_90th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_num_ops=1 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_max=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_mean=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_max=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_min=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRowActionRequestCount=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMutateRequestCount=8 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheCount=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_25th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationBytes=111031 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_mean=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataHitCount=17 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:zeroCopyBytesRead=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_98th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionCount=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_num_ops=1 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileCount=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_num_ops=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_median=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALSize=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_num_ops=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_mean=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_median=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheFreeSize=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:averageRegionSize=32082 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_95th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaMissCount=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_75th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:filteredReadRequestCount=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaHitCount=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_75th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_num_ops=1 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_95th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_median=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCount=18 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_num_ops=0 2023-10-15T16:24:16,737 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.serverName=8c5d72df62de,35419,1697385888915 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReads=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowAppendCount=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:numReferenceFiles=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedFromMob=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSize=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_max=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_90th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_median=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_mean=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_98th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadWins=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseWarnThresholdExceeded=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_max=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_25th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoHitCount=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_median=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreHeapSize=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_25th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_min=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99.9th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_25th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_num_ops=1 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_90th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_max=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99.9th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_max=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_median=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsSize=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_median=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_median=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_min=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkMissCount=1 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_95th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFreeSize=922061632 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheHitPercent=0.0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_98th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_num_ops=2 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.walReaderEditsBufferUsage=80117 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_25th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfHFileRefsQueue=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logReadInBytes=83847 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_75th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfLogQueue=-3 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_90th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99.9th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_num_ops=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedOps=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_min=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedHFiles=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_min=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_mean=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBytes=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_90th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedOps=1000 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedBatches=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_98th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_max=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_95th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedBatches=1 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_mean=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_95th_percentile=0 2023-10-15T16:24:16,738 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedHFiles=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_median=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.uncleanlyClosedLogs=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99th_percentile=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.closedLogsWithUnknownFileLength=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_max=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.repeatedLogFileBytes=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedRecoverQueues=5 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Context=regionserver 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedRecoverQueues=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.restartedLogReading=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.ignoredUncleanlyClosedLogContentsInBytes=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsRead=40 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.numInitializing=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99.9th_percentile=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_75th_percentile=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsFiltered=38 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_25th_percentile=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedLogs=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.failedBatches=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBatches=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_median=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWrite=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosNumOps=26 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRemoved=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsNumOps=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksCached=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsAvgTime=94.0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalNumOps=9 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesRead=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpAvgTime=6326.928571428572 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesAvgTime=0.0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionTasks=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncCount=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:NumProcessedCommands=10 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpNumOps=27 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromLocalClient=28 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:VolumeFailures=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosNumOps=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesRead=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosAvgTime=561936.0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalReadTime=9 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesRead=52810 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsAvgTime=1.7142857142857142 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromRemoteClient=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksUncached=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DatanodeNetworkErrors=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromLocalClient=27 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksInPendingIBR=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockVerificationFailures=0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,739 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosNumOps=54 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesWrite=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosNumOps=54 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.SessionId=null 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsNumOps=1 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesWritten=620489 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRead=27 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReplicated=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Context=dfs 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksVerified=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromRemoteClient=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosAvgTime=20303.65 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsNumOps=9 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksWritten=28 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsAvgTime=1.0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalWriteTime=12 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcDecodingTimeNanos=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesNumOps=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosNumOps=1285 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpNumOps=28 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosAvgTime=23635.714285714286 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesWritten=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosAvgTime=10588.8 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpNumOps=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumFailedVolumes=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheCapacity=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheUsed=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:FailedStorageLocations=[Ljava.lang.String;@530e5ae8 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksCached=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:LastVolumeFailureDate=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Capacity=1054589059072 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToUncache=10 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:DfsUsed=1039261 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Remaining=941153846693 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToCache=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numRegionSizeReports=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_min=0 2023-10-15T16:24:16,740 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_num_ops=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_98th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_num_ops=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numNamespaceInQuotaViolation=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_75th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_95th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_90th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_median=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_max=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.namespaceSpaceQuotaOverview=[] 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numTablesInQuotaViolation=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_num_ops=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99.9th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_max=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_98th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_median=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_min=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_max=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Context=master 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99.9th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_95th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_median=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_mean=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_mean=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_max=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_25th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_mean=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_min=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_75th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_mean=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_90th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_min=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_median=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.tableSpaceQuotaOverview=[] 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numSpaceQuotas=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_25th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_num_ops=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenSubmittedCount=14 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignSubmittedCount=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_num_ops=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99.9th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_median=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_max=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_95th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_95th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_min=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_mean=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_min=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_median=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_min=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveSubmittedCount=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_25th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_mean=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99.9th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_max=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnFS=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerRegions=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_75th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_95th_percentile=0 2023-10-15T16:24:16,741 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignSubmittedCount=13 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_75th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_98th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_num_ops=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_90th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignFailedCount=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_num_ops=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_90th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignFailedCount=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_25th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_98th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_98th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveFailedCount=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_mean=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_75th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeFailedCount=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Context=master 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_num_ops=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseSubmittedCount=1 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:holes=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_75th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99.9th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_95th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_95th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeSubmittedCount=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99.9th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_median=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99.9th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_mean=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCount=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenSubmittedCount=1 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseFailedCount=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_max=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:inconsistentRegions=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_90th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenFailedCount=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnRS=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99.9th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_75th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_75th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_min=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_25th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerOpenRegions=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_95th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_25th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:overlaps=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenFailedCount=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_90th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_98th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitFailedCount=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_max=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_98th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_mean=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_num_ops=13 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_median=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_mean=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCountOverThreshold=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_min=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_75th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_90th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_num_ops=1 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_75th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_95th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_25th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_median=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_25th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitSubmittedCount=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_mean=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritOldestAge=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:emptyRegionInfoRegions=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_max=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_25th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99.9th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_90th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_min=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99.9th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_num_ops=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_max=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_95th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_median=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_max=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_90th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_25th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_25th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_min=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_90th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_95th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_max=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_num_ops=1 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_min=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:operationCount=15 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_98th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:deadServerOpenRegions=3 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_mean=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_median=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_min=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99.9th_percentile=0 2023-10-15T16:24:16,742 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_98th_percentile=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_median=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_max=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_mean=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_num_ops=14 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_98th_percentile=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_90th_percentile=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_median=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_98th_percentile=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_75th_percentile=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumFailedVolumes=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheUsed=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheCapacity=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksCached=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:LastVolumeFailureDate=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Capacity=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Context=FSDatasetState 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:DfsUsed=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Remaining=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToCache=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToUnCache=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalDataFileIos=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalFileIoErrors=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateNumOps=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateNumOps=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateNumOps=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateNumOps=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateNumOps=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateNumOps=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Context=dfs 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalMetadataOperations=0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsNumOps=17 2023-10-15T16:24:16,743 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportNumOps=14 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsNumOps=57 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoNumOps=707 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestNumOps=3 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedNumOps=172 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateNumOps=64 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.port=39027 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeNumOps=3 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportAvgTime=1.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Context=rpcdetailed 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameNumOps=44 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportAvgTime=0.21428571428571427 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteAvgTime=0.25 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateAvgTime=5.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteNumOps=64 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingNumOps=219 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsNumOps=334 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportNumOps=4 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeAvgTime=0.6666666666666666 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionNumOps=1 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockNumOps=53 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteNumOps=41 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateNumOps=1 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesNumOps=16 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeNumOps=1 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatNumOps=1200 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseNumOps=39 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseNumOps=5 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatAvgTime=0.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemMaxM=2200.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTerminated=0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsWaiting=87 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.SessionId=null 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Context=jvm 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapUsedM=193.45564 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTotalExtraSleepTime=55 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Old Generation=0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCount=11 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsBlocked=0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogWarn=0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsRunnable=84 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogInfo=0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsNew=0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapUsedM=566.47656 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTimedWaiting=118 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapMaxM=2200.0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Young Generation=11 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogError=0 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.ProcessName=DataNode 2023-10-15T16:24:16,744 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogFatal=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillis=328 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDataNodeMessageCount=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastWrittenTransactionId=466 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:SnapshottableDirectories=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEncryptionZones=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReconstructionBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastCheckpointTime=1697385856136 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ReadLockLongHoldCount=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumActiveClients=1 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastLogRoll=466 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumTimedOutPendingReconstructions=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:VolumeFailuresTotal=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ProvidedCapacityTotal=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingECBlockGroups=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedGB=0.0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecommissioningDataNodes=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.HAState=active 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyECBlockGroups=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:FilesTotal=109 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ScheduledReplicationBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureECBlockGroups=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptECBlockGroups=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MillisSinceLastLoadedEdits=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalECBlockGroups=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExcessBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalLoad=9 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExpiredHeartbeats=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInServiceLiveDataNodes=3 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumStaleStorages=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:WriteLockLongHoldCount=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlocksTotal=43 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:UnderReplicatedBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicatedBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsed=3126472 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalReplicatedBlocks=43 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:StaleDataNodes=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumFilesUnderConstruction=1 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemaining=2823484494063 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PostponedMisreplicatedBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedNonDFS=178715290424 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LockQueueLength=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptReplicatedBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Context=dfs 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomDeadDataNodes=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:Snapshots=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionECBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.TotalSyncTimes=11 5 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReplicationBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotal=3163767177216 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDeadDataNodes=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplOneBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalSyncCount=417 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastCheckpoint=466 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicationOneBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumLiveDataNodes=3 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlockCapacity=4194304 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotalGB=2946.0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyBlocks=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomLiveDataNodes=0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemainingGB=2630.0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:ReceivedBytes=15456 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:16,745 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcSlowCalls=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationSuccesses=46 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumDroppedConnections=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcClientBackoff=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationFailures=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumOpenConnections=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.port=35085 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:SentBytes=9822 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeNumOps=46 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationSuccesses=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:CallQueueLength=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Context=rpc 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationFailures=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeNumOps=46 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeNumOps=46 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:receivedBytes=288613 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationSuccesses=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_max=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFailures=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numOpenConnections=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_num_ops=1195 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_median=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationSuccesses=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_num_ops=1195 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_num_ops=1195 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_max=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_max=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_median=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.otherExceptions=5 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Context=regionserver 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_median=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_min=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_median=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_max=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:24:16,746 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_min=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_min=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveHandler=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_num_ops=1195 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_mean=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReadHandler=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_median=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_num_ops=1195 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_mean=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callDropped=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions=5 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFallbacks=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationFailures=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_min=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_max=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:queueSize=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveScanHandler=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_min=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:sentBytes=44933 2023-10-15T16:24:16,747 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_mean=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_75th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_max=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_90th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_min=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_median=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_num_ops=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOnHeapSize=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_num_ops=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_num_ops=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_median=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_num_ops=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_75th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_min=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOffHeapSize=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_25th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_75th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_mean=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_25th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_25th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_mean=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tunerDoNothingCounter=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_25th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_num_ops=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:aboveHeapOccupancyLowWaterMarkCounter=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_98th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreSize=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_min=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_max=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_90th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_mean=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_75th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Context=regionserver 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99.9th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_mean=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_min=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockCacheSize=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_90th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockedFlushGauge=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_max=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_median=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_98th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:unblockedFlushGauge=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_max=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_max=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_num_ops=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_min=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_95th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_90th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_95th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_max=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_median=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_98th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_min=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99.9th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_median=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_median=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_95th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_mean=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_95th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_98th_percentile=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalDataFileIos=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalFileIoErrors=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateNumOps=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateNumOps=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateNumOps=0 2023-10-15T16:24:16,748 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateNumOps=0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateNumOps=0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateNumOps=0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Context=dfs 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalMetadataOperations=0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:HttpPort=null 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XceiverCount=3 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"2","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-15... 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:ClusterId=testClusterID 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test... 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DatanodeHostname=127.0.0.1 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SecurityEnabled=false 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:RpcPort=0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DataPort=0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SoftwareVersion=3.2.4 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SlowDisks=null 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XmitsInProgress=0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DNStartedTimeInMillis=1697385857021 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsBatchedInSync=50 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsNumOps=416 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsAvgTime=0.0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportNumOps=0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeNumOps=0 2023-10-15T16:24:16,749 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteSnapshotOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsAvgTime=0.05714285714285714 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AllowSnapshotOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:RenameSnapshotOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeAvgTime=0.05 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesCreated=176 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesAppended=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportAvgTime=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageAvgTime=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeNumOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeNumOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeNumOps=246 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeAvgTime=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumTimesReReplicationNotScheduled=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SafeModeTime=1 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalNumOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetListingOps=138 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.ProcessName=NameNode 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageNumOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TotalFileOps=1372 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSymlinkOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AddBlockOps=53 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DisallowSnapshotOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteFileOps=31 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsNumOps=466 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSnapshotOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PendingEditsCount=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ListSnapshottableDirOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockReceivedAndDeletedOps=172 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.SessionId=null 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditNumOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FsImageLoadTime=23 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateFileOps=64 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Context=dfs 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesRenamed=44 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TimeoutReReplications=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsQueued=1 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedNumOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageNumOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageAvgTime=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesInGetListingOps=154 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FileInfoOps=707 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetLinkTargetOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetAdditionalDatanodeOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsBatched=71 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesDeleted=47 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SuccessfulReReplications=23 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetBlockLocations=334 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesTruncated=1 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SnapshotDiffReportOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportNumOps=8 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeNumOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditAvgTime=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportAvgTime=0.5 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumFailedVolumes=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheUsed=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheCapacity=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksCached=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:LastVolumeFailureDate=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Capacity=1054589059072 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Context=FSDatasetState 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-... 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:DfsUsed=1039261 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Remaining=941156091301 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToCache=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToUnCache=10 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalDataFileIos=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalFileIoErrors=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateNumOps=0 2023-10-15T16:24:16,750 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateNumOps=0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateNumOps=0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateNumOps=0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateNumOps=0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateNumOps=0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Context=dfs 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalMetadataOperations=0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessNumOps=1 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureNumOps=0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailures=0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsNumOps=4 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailuresTotal=0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Context=ugi 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsAvgTime=29.0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessAvgTime=10.0 2023-10-15T16:24:16,751 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureAvgTime=0.0 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfSnapshottableDirs=0 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheCapacity=0 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheUsed=0 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ProvidedCapacity=0 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DistinctVersionCount=1 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocksWithReplicationFactorOne=0 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentRemaining=89.244385 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentUsed=9.8821176E-5 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:TotalBlocks=43 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NonDfsUsedSpace=178715290424 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Safemode= 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:UpgradeFinalized=true 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:EnteringMaintenanceNodes={} 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentBlockPoolUsed=9.8821176E-5 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:LiveNodes={"127.0.0.1:40557":{"infoAddr":"127.0.0.1:35357","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:40557","lastContact":2,"us... 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CompileInfo=2022-07-12T11:58Z by ubuntu from branch-3.2.4 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolId=BP-1531809096-172.17.0.3-1697385856100 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ClusterId=testClusterID 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DeadNodes={} 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:JournalTransactionInfo={"MostRecentCheckpointTxId":"0","LastAppliedOrWrittenTxId":"466"} 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Free=2823484494063 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolUsedSpace=3126472 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Threads=289 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocks=0 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirStatuses={"active":{"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/t... 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:SoftwareVersion=3.2.4 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:VerifyECWithTopologyResult={"isSupported":"false","resultMessage":"The number of DataNodes (3) is less than the minimum required number of DataNodes (9) fo... 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Used=3126472 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Total=3163767177216 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NodeUsage={"nodeUsage":{"min":"0.00%","median":"0.00%","max":"0.00%","stdDev":"0.00%"}} 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NNStartedTimeInMillis=1697385856352 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DecomNodes={} 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CorruptFiles=[] 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirSize={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test... 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameJournalStatus=[{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3... 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemMaxM=2200.0 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,752 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTerminated=0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsWaiting=87 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.SessionId=null 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Context=jvm 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapUsedM=193.46402 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTotalExtraSleepTime=55 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Old Generation=0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCount=11 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsBlocked=0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogWarn=0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsRunnable=84 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogInfo=0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsNew=0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapUsedM=567.47656 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTimedWaiting=118 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapMaxM=2200.0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Young Generation=11 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogError=0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.ProcessName=DataNode 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogFatal=0 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillis=328 2023-10-15T16:24:16,753 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlocksTotal=43 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TopUserOpCounts={"timestamp":"2023-10-15T16:24:16+0000","windows":[{"ops":[{"opType":"*","totalCount":0,"allUsers":[],"topUsers":[]}],"windowLen... 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:SnapshotStats={"SnapshottableDirectories":0,"Snapshots":0} 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:UnderReplicatedBlocks=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityUsed=3126472 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEncryptionZones=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityRemaining=2823484494063 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReconstructionBlocks=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomDeadDataNodes=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FSState=Operational 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:VolumeFailuresTotal=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ProvidedCapacityTotal=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecommissioningDataNodes=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReplicationBlocks=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncTimes=11 5 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingDeletionBlocks=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityTotal=3163767177216 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FilesTotal=109 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ScheduledReplicationBlocks=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDeadDataNodes=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncCount=417 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FsLockQueueLength=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:MaxObjects=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalLoad=9 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleDataNodes=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumLiveDataNodes=3 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlockDeletionStartTime=1697385856352 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInServiceLiveDataNodes=3 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:LowRedundancyBlocks=0 2023-10-15T16:24:16,754 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomLiveDataNodes=0 2023-10-15T16:24:16,755 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleStorages=0 2023-10-15T16:24:16,755 INFO [pool-242-thread-1 {}] common.MetricsLoggerTask(109): << End NameNode metrics dump 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(83): >> Begin DataNode metrics dump 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_num_ops=0 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_mean=0 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_num_ops=0 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_25th_percentile=0 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_95th_percentile=0 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_90th_percentile=0 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_90th_percentile=0 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_max=0 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_98th_percentile=0 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_25th_percentile=0 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_75th_percentile=0 2023-10-15T16:24:16,787 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_min=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_75th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_min=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_max=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_max=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_25th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99.9th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_75th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_mean=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99.9th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_75th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_median=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_95th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_98th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99.9th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_median=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_25th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Context=master 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_median=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_95th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_98th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_90th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_mean=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99.9th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_95th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_min=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_max=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_98th_percentile=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_mean=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_num_ops=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_min=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_median=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_num_ops=0 2023-10-15T16:24:16,788 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_90th_percentile=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:HttpPort=null 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XceiverCount=3 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"2","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"384","LastBlockReport":"485"}] 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:ClusterId=testClusterID 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1":{"numBlocks":21,"usedSpace":259482,"freeSpace":470509548965,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":134216283},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2":{"numBlocks":22,"usedSpace":779779,"freeSpace":470643765248,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0}} 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DatanodeHostname=127.0.0.1 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SecurityEnabled=false 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:RpcPort=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DataPort=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SoftwareVersion=3.2.4 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SlowDisks=null 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XmitsInProgress=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DNStartedTimeInMillis=1697385856691 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Context=regionserver 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.port=44717 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Context=rpcdetailed 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthNumOps=50 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpNumOps=5 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWrite=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosNumOps=48 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRemoved=10 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalAvgTime=0.8367346938775511 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsNumOps=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksCached=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsAvgTime=2.0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DataNodeActiveXceiversCount=1 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalNumOps=400 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsNumOps=58 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesRead=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpAvgTime=50.0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpAvgTime=225065.0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesAvgTime=0.0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionTasks=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncCount=0 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:NumProcessedCommands=401 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpNumOps=104 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromLocalClient=52 2023-10-15T16:24:16,791 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:VolumeFailures=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosNumOps=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesRead=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosAvgTime=818128.0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalReadTime=18 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesRead=362275 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsAvgTime=1.4 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromRemoteClient=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksUncached=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DatanodeNetworkErrors=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromLocalClient=104 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksInPendingIBR=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockVerificationFailures=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosNumOps=208 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesWrite=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosNumOps=208 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.SessionId=null 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsNumOps=1 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesWritten=1008141 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRead=104 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReplicated=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Context=dfs 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksVerified=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromRemoteClient=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosAvgTime=26277.0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsNumOps=400 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksWritten=52 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsAvgTime=0.7857142857142857 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalWriteTime=21 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcDecodingTimeNanos=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesNumOps=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosNumOps=1724 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpNumOps=52 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosAvgTime=5329.0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesWritten=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosAvgTime=10111.5 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpAvgTime=0.0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpNumOps=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumFailedVolumes=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheUsed=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheCapacity=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksCached=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:LastVolumeFailureDate=0 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Capacity=1054589059072 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Context=FSDatasetState 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:DfsUsed=1039261 2023-10-15T16:24:16,792 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Remaining=941156091301 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToCache=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToUnCache=10 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheCleared=1 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheHit=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Context=rpc 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheUpdated=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalDataFileIos=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalFileIoErrors=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateNumOps=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateNumOps=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateNumOps=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateNumOps=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateNumOps=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateNumOps=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Context=dfs 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalMetadataOperations=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:MissingECBlockGroups=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:LowRedundancyECBlockGroups=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:BytesInFutureECBlockGroups=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:CorruptECBlockGroups=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:PendingDeletionECBlocks=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:TotalECBlockGroups=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheCleared=1 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheHit=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Context=rpc 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheUpdated=0 2023-10-15T16:24:16,793 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:LastHATransitionTime=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:HostAndPort=localhost:39027 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:State=active 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowPeersReport=null 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowDisksReport=null 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:NNRole=NameNode 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:BytesWithFutureGenerationStamps=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SecurityEnabled=false 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.port=35753 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Context=rpcdetailed 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthNumOps=55 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:ReceivedBytes=18480 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcSlowCalls=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationSuccesses=55 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumDroppedConnections=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcClientBackoff=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationFailures=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumOpenConnections=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.port=35753 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:SentBytes=11751 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeNumOps=55 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationSuccesses=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:CallQueueLength=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Context=rpc 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationFailures=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeNumOps=55 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeNumOps=55 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:ReceivedBytes=16800 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcSlowCalls=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationSuccesses=50 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumDroppedConnections=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcClientBackoff=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationFailures=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumOpenConnections=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.port=44717 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:SentBytes=10673 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeNumOps=50 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationSuccesses=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:CallQueueLength=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Context=rpc 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationFailures=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeNumOps=50 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeNumOps=50 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemMaxM=2200.0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTerminated=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsWaiting=87 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.SessionId=null 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Context=jvm 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapUsedM=193.44197 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTotalExtraSleepTime=51 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Old Generation=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCount=11 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsBlocked=0 2023-10-15T16:24:16,794 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogWarn=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsRunnable=84 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogInfo=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsNew=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapUsedM=562.47656 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTimedWaiting=118 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapMaxM=2200.0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Young Generation=11 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogError=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.ProcessName=NameNode 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogFatal=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillis=328 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumFailedVolumes=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheUsed=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheCapacity=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksCached=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:LastVolumeFailureDate=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Capacity=1054589059072 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Context=FSDatasetState 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:DfsUsed=1047950 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Remaining=941156091301 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToCache=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToUnCache=10 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemMaxM=2200.0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTerminated=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsWaiting=87 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.SessionId=null 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Context=jvm 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapUsedM=193.44412 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTotalExtraSleepTime=22 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Old Generation=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCount=11 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsBlocked=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogWarn=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsRunnable=84 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogInfo=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsNew=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapUsedM=562.47656 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTimedWaiting=118 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapMaxM=2200.0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Young Generation=11 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogError=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.ProcessName=DataNode 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogFatal=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillis=328 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_num_ops=1906 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99th_percentile=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_max=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_mean=187 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:appendCount=1906 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:lowReplicaRollRequest=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99th_percentile=336 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99.9th_percentile=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:sizeRollRequest=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendSize=301542 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_max=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowAppendCount=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:successfulLogRolls=25 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99.9th_percentile=337 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_TimeRangeCount_0-1=4 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_25th_percentile=37 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendSize=9557 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_90th_percentile=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendCount=1011 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_25th_percentile=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_10-100=2 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99th_percentile=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_98th_percentile=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_75th_percentile=297 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_75th_percentile=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_max=338 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_95th_percentile=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_mean=0 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendCount=48 2023-10-15T16:24:16,795 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_100-1000=2 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:errorRollRequest=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:rollRequest=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_num_ops=1819 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_median=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_98th_percentile=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_min=36 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_min=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_median=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Context=regionserver 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_0-1=4 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendCount=45 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_95th_percentile=329 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendSize=74104 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_median=38 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_98th_percentile=334 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_25th_percentile=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_75th_percentile=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_mean=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99.9th_percentile=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendSize=14494 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:writtenBytes=399697 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_90th_percentile=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_95th_percentile=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowSyncRollRequest=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_num_ops=1906 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_90th_percentile=321 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_min=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendCount=802 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicationOneBlocks=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:CorruptReplicatedBlocks=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicatedBlocks=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:TotalReplicatedBlocks=43 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalDataFileIos=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalFileIoErrors=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateNumOps=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateNumOps=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateNumOps=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateNumOps=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateNumOps=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateNumOps=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Context=dfs 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalMetadataOperations=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotNumOps=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishNumOps=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotAvgTime=0.0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Context=metricssystem 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSources=67 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:DroppedPubAll=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishAvgTime=0.0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSources=67 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSinks=0 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,796 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSinks=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheCleared=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheHit=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Context=rpc 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheUpdated=149 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWrite=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosNumOps=28 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRemoved=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsNumOps=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksCached=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsAvgTime=3.0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalNumOps=9 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesRead=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpAvgTime=9.0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpAvgTime=6810.7692307692305 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesAvgTime=0.0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionTasks=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncCount=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:NumProcessedCommands=10 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpNumOps=31 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromLocalClient=28 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:VolumeFailures=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosNumOps=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesRead=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosAvgTime=554484.75 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalReadTime=7 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesRead=57958 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsAvgTime=1.5714285714285714 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromRemoteClient=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksUncached=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DatanodeNetworkErrors=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromLocalClient=31 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksInPendingIBR=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockVerificationFailures=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosNumOps=62 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesWrite=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosNumOps=62 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.SessionId=null 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsNumOps=1 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesWritten=620489 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRead=31 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReplicated=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Context=dfs 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksVerified=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromRemoteClient=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosAvgTime=19658.18181818182 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsNumOps=9 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,797 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksWritten=28 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsAvgTime=1.0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalWriteTime=12 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcDecodingTimeNanos=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesNumOps=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosNumOps=1285 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpNumOps=28 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosAvgTime=4301.205882352941 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesWritten=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosAvgTime=9971.454545454546 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpAvgTime=0.8181818181818182 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpNumOps=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemMaxM=2200.0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTerminated=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsWaiting=87 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.SessionId=null 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Context=jvm 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapUsedM=193.44412 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTotalExtraSleepTime=62 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Old Generation=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCount=11 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsBlocked=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogWarn=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsRunnable=84 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogInfo=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsNew=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapUsedM=562.47656 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTimedWaiting=118 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapMaxM=2200.0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Young Generation=11 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogError=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.ProcessName=DataNode 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogFatal=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillis=328 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Context=regionserver 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:numTables=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99.9th_percentile=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_90th_percentile=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_95th_percentile=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterRequests=1101 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99th_percentile=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterWriteRequests=1059 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_mean=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterReadRequests=74 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_min=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_25th_percentile=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashSubmittedCount=4 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashFailedCount=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_98th_percentile=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Context=master 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_num_ops=2 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_75th_percentile=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_max=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_median=0 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:receivedBytes=6051329 2023-10-15T16:24:16,798 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_98th_percentile=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationSuccesses=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_25th_percentile=16 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_max=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_98th_percentile=16 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_90th_percentile=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFailures=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_98th_percentile=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numOpenConnections=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_75th_percentile=16 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_num_ops=1334 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99.9th_percentile=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_median=36 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_90th_percentile=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationSuccesses=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99.9th_percentile=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_max=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_num_ops=1334 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_num_ops=1334 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_max=16 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_max=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_median=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.otherExceptions=3 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_95th_percentile=16 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Context=master 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99th_percentile=16 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_median=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_min=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99.9th_percentile=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_median=16 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_max=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_95th_percentile=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_min=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_98th_percentile=36 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_min=16 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_SizeRangeCount_10-100=30 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99.9th_percentile=16 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99th_percentile=1 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveHandler=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_num_ops=1334 2023-10-15T16:24:16,799 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_mean=16 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_SizeRangeCount_10-100=30 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReadHandler=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_median=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_90th_percentile=36 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_num_ops=1334 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99th_percentile=1 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_mean=36 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_95th_percentile=1 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_25th_percentile=36 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callDropped=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_75th_percentile=36 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions=3 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFallbacks=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99th_percentile=1 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationFailures=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_min=36 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_95th_percentile=1 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_max=36 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:queueSize=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99.9th_percentile=36 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveScanHandler=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_95th_percentile=36 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_98th_percentile=1 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_90th_percentile=16 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99th_percentile=36 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_min=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:sentBytes=18267 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumFailedVolumes=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheUsed=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheCapacity=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksCached=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:LastVolumeFailureDate=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Capacity=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Context=FSDatasetState 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data4]'} 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:DfsUsed=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Remaining=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToCache=0 2023-10-15T16:24:16,800 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToUnCache=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:HttpPort=null 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XceiverCount=3 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"1","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"1198"}] 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:ClusterId=testClusterID 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5":{"numBlocks":22,"usedSpace":274432,"freeSpace":470509356453,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":134216283},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6":{"numBlocks":21,"usedSpace":773518,"freeSpace":470643572736,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0}} 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DatanodeHostname=127.0.0.1 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SecurityEnabled=false 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:RpcPort=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DataPort=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SoftwareVersion=3.2.4 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SlowDisks=null 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XmitsInProgress=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DNStartedTimeInMillis=1697385857417 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalDataFileIos=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalFileIoErrors=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateNumOps=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateNumOps=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateNumOps=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateNumOps=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateNumOps=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateNumOps=0 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Context=dfs 2023-10-15T16:24:16,801 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalMetadataOperations=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumFailedVolumes=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheUsed=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheCapacity=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksCached=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:LastVolumeFailureDate=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Capacity=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Context=FSDatasetState 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data6]'} 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:DfsUsed=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Remaining=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToCache=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToUnCache=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:ReceivedBytes=1072216 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcSlowCalls=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationSuccesses=47 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumDroppedConnections=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.NumOpenConnectionsPerUser={"jenkins":1} 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcClientBackoff=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationFailures=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumOpenConnections=1 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.port=39027 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:SentBytes=679332 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeNumOps=3059 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationSuccesses=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:CallQueueLength=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Context=rpc 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationFailures=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeNumOps=3059 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.serverName=ClientNamenodeProtocol 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeNumOps=3059 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.port=35085 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Context=rpcdetailed 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthNumOps=46 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemMaxM=2200.0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTerminated=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsWaiting=87 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.SessionId=null 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Context=jvm 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapUsedM=193.44489 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTotalExtraSleepTime=45 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Old Generation=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCount=11 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsBlocked=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogWarn=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsRunnable=84 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogInfo=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsNew=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapUsedM=562.47656 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTimedWaiting=118 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapMaxM=2200.0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Young Generation=11 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogError=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.ProcessName=DataNode 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogFatal=0 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillis=328 2023-10-15T16:24:16,802 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsBatchedInSync=31 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsNumOps=224 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsAvgTime=0.13043478260869565 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportNumOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeNumOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteSnapshotOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsAvgTime=0.03896103896103896 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AllowSnapshotOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:RenameSnapshotOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeAvgTime=0.0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesCreated=80 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesAppended=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportAvgTime=0.0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageAvgTime=0.0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeNumOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeNumOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeNumOps=3 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeAvgTime=0.0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumTimesReReplicationNotScheduled=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SafeModeTime=3 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalNumOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetListingOps=66 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.ProcessName=NameNode 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageNumOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TotalFileOps=540 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSymlinkOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AddBlockOps=28 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DisallowSnapshotOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteFileOps=5 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsNumOps=255 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSnapshotOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PendingEditsCount=1 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ListSnapshottableDirOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockReceivedAndDeletedOps=96 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.SessionId=null 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditNumOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FsImageLoadTime=173 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateFileOps=36 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Context=dfs 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesRenamed=17 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TimeoutReReplications=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsQueued=1 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedNumOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageNumOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageAvgTime=0.0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesInGetListingOps=35 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FileInfoOps=270 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetLinkTargetOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetAdditionalDatanodeOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsBatched=9 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesDeleted=1 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SuccessfulReReplications=2 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetBlockLocations=118 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesTruncated=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SnapshotDiffReportOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportNumOps=6 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeNumOps=0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditAvgTime=0.0 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportAvgTime=0.5 2023-10-15T16:24:16,803 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumFailedVolumes=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheCapacity=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheUsed=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:FailedStorageLocations=[Ljava.lang.String;@59b52a36 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksCached=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:LastVolumeFailureDate=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Capacity=1054589059072 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToUncache=10 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:DfsUsed=1039261 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Remaining=941152798117 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToCache=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpNumOps=6 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWrite=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosNumOps=54 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRemoved=10 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalAvgTime=0.7474747474747475 2023-10-15T16:24:16,805 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsNumOps=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksCached=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsAvgTime=2.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DataNodeActiveXceiversCount=1 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalNumOps=400 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsNumOps=55 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesRead=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpAvgTime=34.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpAvgTime=225066.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesAvgTime=0.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionTasks=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncCount=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:NumProcessedCommands=401 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpNumOps=99 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromLocalClient=52 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:VolumeFailures=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosNumOps=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesRead=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosAvgTime=924152.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalReadTime=29 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesRead=688752 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsAvgTime=2.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromRemoteClient=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksUncached=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DatanodeNetworkErrors=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromLocalClient=99 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksInPendingIBR=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockVerificationFailures=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosNumOps=200 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesWrite=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosNumOps=200 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.SessionId=null 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsNumOps=1 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesWritten=1008141 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRead=99 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReplicated=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Context=dfs 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksVerified=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromRemoteClient=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosAvgTime=45642.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsNumOps=400 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksWritten=52 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsAvgTime=0.696969696969697 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalWriteTime=22 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcDecodingTimeNanos=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesNumOps=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosNumOps=1724 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpNumOps=52 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosAvgTime=6378.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesWritten=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosAvgTime=9294.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpNumOps=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,806 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWrite=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosNumOps=30 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRemoved=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsNumOps=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksCached=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsAvgTime=95.0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalNumOps=9 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesRead=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpAvgTime=6809.0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesAvgTime=0.0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionTasks=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncCount=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:NumProcessedCommands=10 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpNumOps=35 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromLocalClient=28 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:VolumeFailures=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosNumOps=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesRead=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosAvgTime=352052.375 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalReadTime=16 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesRead=75811 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsAvgTime=1.3846153846153846 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromRemoteClient=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksUncached=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DatanodeNetworkErrors=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromLocalClient=35 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksInPendingIBR=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockVerificationFailures=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosNumOps=70 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesWrite=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosNumOps=70 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.SessionId=null 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsNumOps=1 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesWritten=620489 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRead=35 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReplicated=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Context=dfs 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksVerified=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromRemoteClient=0 2023-10-15T16:24:16,807 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosAvgTime=25843.916666666668 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsNumOps=9 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksWritten=28 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsAvgTime=1.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalWriteTime=12 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcDecodingTimeNanos=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesNumOps=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosNumOps=1285 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpNumOps=28 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosAvgTime=4740.882352941177 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesWritten=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosAvgTime=9895.041666666666 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpAvgTime=0.75 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpNumOps=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpNumOps=6 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWrite=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosNumOps=34 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRemoved=10 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalAvgTime=0.8877551020408163 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsNumOps=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksCached=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsAvgTime=3.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DataNodeActiveXceiversCount=1 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalNumOps=400 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsNumOps=59 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesRead=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpAvgTime=10.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpAvgTime=225063.25 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesAvgTime=0.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionTasks=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncCount=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:NumProcessedCommands=402 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpNumOps=91 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromLocalClient=52 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:VolumeFailures=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosNumOps=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesRead=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosAvgTime=1214340.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalReadTime=12 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesRead=349472 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsAvgTime=1.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromRemoteClient=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksUncached=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DatanodeNetworkErrors=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromLocalClient=91 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksInPendingIBR=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockVerificationFailures=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosNumOps=182 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesWrite=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosNumOps=182 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.SessionId=null 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsNumOps=2 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesWritten=1008141 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRead=91 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReplicated=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Context=dfs 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksVerified=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromRemoteClient=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,808 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosAvgTime=53154.75 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsNumOps=400 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksWritten=52 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsAvgTime=0.8571428571428571 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalWriteTime=28 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcDecodingTimeNanos=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesNumOps=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosNumOps=1724 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpNumOps=52 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosAvgTime=5041.5 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesWritten=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosAvgTime=8707.0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpNumOps=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDataNodeMessageCount=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastWrittenTransactionId=255 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:SnapshottableDirectories=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEncryptionZones=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReconstructionBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastCheckpointTime=1697385845476 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ReadLockLongHoldCount=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumActiveClients=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastLogRoll=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumTimedOutPendingReconstructions=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:VolumeFailuresTotal=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ProvidedCapacityTotal=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingECBlockGroups=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedGB=0.0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecommissioningDataNodes=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.HAState=active 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyECBlockGroups=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:FilesTotal=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ScheduledReplicationBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureECBlockGroups=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptECBlockGroups=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MillisSinceLastLoadedEdits=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalECBlockGroups=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExcessBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalLoad=32 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExpiredHeartbeats=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInServiceLiveDataNodes=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumStaleStorages=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:WriteLockLongHoldCount=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlocksTotal=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:UnderReplicatedBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicatedBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsed=495713 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalReplicatedBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:StaleDataNodes=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumFilesUnderConstruction=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemaining=2816043276636 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PostponedMisreplicatedBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedNonDFS=185085906847 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LockQueueLength=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptReplicatedBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Context=dfs 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomDeadDataNodes=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:Snapshots=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionECBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.TotalSyncTimes= 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReplicationBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotal=3163767177216 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDeadDataNodes=3 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplOneBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalSyncCount=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastCheckpoint=255 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicationOneBlocks=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumLiveDataNodes=0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlockCapacity=4194304 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotalGB=2946.0 2023-10-15T16:24:16,809 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyBlocks=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomLiveDataNodes=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemainingGB=2623.0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Context=regionserver 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheCleared=1 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheHit=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Context=rpc 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheUpdated=65 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemMaxM=2200.0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTerminated=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsWaiting=87 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.SessionId= 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Context=jvm 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapUsedM=193.45454 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Old Generation=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCount=11 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsBlocked=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogWarn=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsRunnable=84 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogInfo=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsNew=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapUsedM=563.47656 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTimedWaiting=118 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapMaxM=2200.0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Young Generation=11 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogError=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.ProcessName=Master 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogFatal=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillis=328 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsCount=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeTotal=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointPercentComplete=1.0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageTotal=1 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsTotal=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointCount=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:ElapsedTime=681 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsPercentComplete=1.0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsElapsedTime=7 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointElapsedTime=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModePercentComplete=1.0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:PercentComplete=1.0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointTotal=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeCount=0 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeElapsedTime=13 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageElapsedTime=62 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageCount=1 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,810 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImagePercentComplete=1.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumFailedVolumes=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheCapacity=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheUsed=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:FailedStorageLocations=[Ljava.lang.String;@12584c42 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksCached=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:LastVolumeFailureDate=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Capacity=1054589059072 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToUncache=10 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:DfsUsed=1047950 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Remaining=941152535973 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToCache=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsCount=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeTotal=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointPercentComplete=1.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageTotal=1 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsTotal=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointCount=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:ElapsedTime=681 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsPercentComplete=1.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsElapsedTime=7 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointElapsedTime=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModePercentComplete=1.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:PercentComplete=1.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointTotal=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeCount=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeElapsedTime=13 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageElapsedTime=62 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageCount=1 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImagePercentComplete=1.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RackLocalityCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionCountSkewCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaHostCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ServerLocalityCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MemStoreSizeCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_95th_percentile=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_Overall=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ReadRequestCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_25th_percentile=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_98th_percentile=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_min=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Context=master 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_WriteRequestCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaRackCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99th_percentile=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_PrimaryRegionCountSkewCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:miscInvocationCount=18 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_median=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_num_ops=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_75th_percentile=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MoveCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_max=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_mean=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_CPRequestCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_TableSkewCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_StoreFileCostFunction=0.0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_90th_percentile=0 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.isBalancerActive=true 2023-10-15T16:24:16,811 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99.9th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_95th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99.9th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_median=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_mean=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_25th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_max=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_95th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_num_ops=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_25th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_98th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_min=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_mean=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_90th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_75th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_90th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_mean=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_num_ops=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_95th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_min=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_median=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_num_ops=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_mean=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_98th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_75th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_median=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99.9th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Context=regionserver 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_max=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_num_ops=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_num_ops=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_min=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_90th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_mean=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_95th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_max=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_75th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_75th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_min=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_num_ops=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_75th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_max=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_min=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_98th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_90th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_max=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_98th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_90th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_75th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_95th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_25th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_max=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_max=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_98th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_num_ops=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99.9th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_min=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_98th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_95th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_min=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_median=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:BlockBytesScannedCount=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99.9th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_25th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_median=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_90th_percentile=0 2023-10-15T16:24:16,812 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_95th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_num_ops=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_max=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_mean=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_max=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_min=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_mean=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_95th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99.9th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_25th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99.9th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_90th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_mean=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_95th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_min=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_75th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_max=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_max=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_90th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_max=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_98th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_25th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_median=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_median=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_75th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_mean=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99.9th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_median=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_90th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_mean=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_mean=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_75th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_num_ops=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99.9th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_median=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_75th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_median=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_median=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_98th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_95th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_75th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_95th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_num_ops=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_max=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_98th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_25th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_95th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99.9th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_num_ops=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_25th_percentile=0 2023-10-15T16:24:16,813 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99.9th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_median=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_90th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_25th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_median=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_min=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_max=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_90th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_max=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_75th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_25th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_98th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_min=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_mean=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99.9th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_25th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_min=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_mean=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_min=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_98th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_min=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_98th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99.9th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_median=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_num_ops=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_median=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_90th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_min=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_25th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,814 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsCount=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_98th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_median=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestRatePerSecond=0.0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerMissCount=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_mean=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_median=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_98th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_95th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_median=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_90th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_5min_rate=0.0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_min=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_95th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitRatio=0.0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_75th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_90th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_num_ops=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_min=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_25th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestCount=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedOutputBytes=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_90th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkHitCount=1 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadOpsInCurThread=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_median=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestRatePerSecond=0.0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_25th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_min=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_num_ops=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_num_ops=26 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_90th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockBytesScannedCount=4389 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_max=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_75th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_max=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_mean=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_num_ops=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_25th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheEvictionCount=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99.9th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheCount=3 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:smallCompactionQueueLength=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCount=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_min=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_num_ops=15 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_mean=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowIncrementCount=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushQueueLength=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_mean=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_75th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:localBytesRead=123986 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushCount=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_median=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutateFailedCount=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_max=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorPoolAllocationBytes=1530880 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_98th_percentile=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:16,815 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_min=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_median=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_min=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:updatesBlockedTime=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_median=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_mean=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitQueueLength=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_min=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_min=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_95th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCachingCount=3 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_25th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99.9th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileAge=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_median=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_75th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_max=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_95th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_num_ops=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_min=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cpRequestCount=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_90th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_median=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_max=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_num_ops=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALCount=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_98th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitRatio=0.7272727272727273 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataMissCount=1 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestCount=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissCount=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_75th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_mean=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheEvictionCount=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheFreeSize=922067336 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_95th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_median=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_75th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_max=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_median=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:activeScanners=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissCount=3 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_mean=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_max=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_max=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_95th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactionQueueLength=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_95th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_num_ops=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_max=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_1min_rate=0.0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_min=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_median=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_num_ops=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_median=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_mean=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_75th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_min=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99.9th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_median=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_25th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_25th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:16,816 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_90th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_num_ops=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheSize=679544 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_num_ops=6 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_num_ops=26 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_98th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_count=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_median=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowGetCount=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_median=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterRequestsCount=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_min=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_min=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_25th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_max=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileSize=836094 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_min=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileCount=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsCount=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99.9th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_98th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_num_ops=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedOutputBytes=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_max=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_num_ops=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_max=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticBloomSize=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexMissCount=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseInfoThresholdExceeded=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCount=3 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCount=3 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_max=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedToMob=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedMemstoreBytes=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheExpressHitPercent=72.72727272727273 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionServerStartTime=1697385859285 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedInputBytes=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedFromMob=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_max=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRequestCount=47 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_25th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcGetRequestCount=3 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_90th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeCount=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCountHitPercent=72.72727272727273 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_max=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_mean=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_min=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_98th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_min=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsSize=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_mean=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowDeleteCount=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockedRequestCount=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_mean=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheAccessCount=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_min=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_median=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_min=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_90th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_min=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_95th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_90th_percentile=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheEvictedCount=0 2023-10-15T16:24:16,817 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_90th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_max=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:avgStoreFileAge=1819 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_25th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheCount=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitRequestCount=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_num_ops=1 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_min=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalBytesRead=123986 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedToMob=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_max=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedInputBytes=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheSize=679544 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileCount=953 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_mean=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcScanRequestCount=26 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreOffHeapSize=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_75th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheMissCount=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowPutCount=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_max=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreSize=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_75th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsSize=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_mean=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_min=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_75th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_median=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_98th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_median=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Context=regionserver 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:scannerLeaseExpiredCount=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:minStoreFileAge=1782 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitCount=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_98th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcFullScanRequestCount=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_mean=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitSuccessCount=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_mean=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_75th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_25th_percentile=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCountPrimary=3 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_max=0 2023-10-15T16:24:16,818 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataBlockCount=1 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_98th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_max=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_median=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsCount=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_mean=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsCount=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCachingCount=8 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFailedInsertionCount=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_90th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsSize=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean_rate=0.0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticIndexSize=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorUsedBufferCount=2 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_min=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99.9th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissRatio=0.2727272727272727 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_75th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_max=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_mean=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_95th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMultiRequestCount=1 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheSize=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexHitCount=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_max=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_98th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedOutputBytes=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocal=0.0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_98th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_max=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_num_ops=3 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99.9th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_25th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_min=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_mean=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_15min_rate=0.0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99.9th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_min=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_95th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_75th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_median=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_median=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_mean=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_95th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitCount=8 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:largeCompactionQueueLength=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_mean=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutatePassedCount=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_max=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_95th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissRatio=0.0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsCount=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_min=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_mean=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_min=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:shortCircuitBytesRead=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99.9th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:16,819 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_median=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_min=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoMissCount=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileIndexSize=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99.9th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_mean=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerHitCount=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_98th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsSize=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCountPrimary=8 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_min=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_max=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99.9th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_90th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_90th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_num_ops=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_max=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_mean=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_max=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_min=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRowActionRequestCount=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMutateRequestCount=16 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheCount=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_25th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_mean=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataHitCount=7 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:zeroCopyBytesRead=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_98th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionCount=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_num_ops=1 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileCount=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_num_ops=1 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_median=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALSize=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_num_ops=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_mean=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_median=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheFreeSize=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:averageRegionSize=17474 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_95th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaMissCount=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_75th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:filteredReadRequestCount=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaHitCount=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_75th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_num_ops=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_95th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_median=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCount=8 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_num_ops=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.serverName=8c5d72df62de,45999,1697385859285 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReads=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowAppendCount=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:numReferenceFiles=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedFromMob=0 2023-10-15T16:24:16,820 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSize=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_max=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationRatio=1.0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_90th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_median=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_mean=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_98th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadWins=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseWarnThresholdExceeded=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_max=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_25th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoHitCount=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_median=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreHeapSize=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_25th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_min=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99.9th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_25th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_num_ops=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_90th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_max=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99.9th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_max=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_median=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsSize=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_median=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_median=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_min=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkMissCount=2 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_95th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFreeSize=922067336 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheHitPercent=0.0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemMaxM=2200.0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTerminated=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsWaiting=87 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.SessionId=null 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Context=jvm 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapUsedM=193.45564 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTotalExtraSleepTime=46 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Old Generation=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCount=11 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsBlocked=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogWarn=0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsRunnable=84 2023-10-15T16:24:16,821 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogInfo=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsNew=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapUsedM=564.47656 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTimedWaiting=118 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapMaxM=2200.0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Young Generation=11 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogError=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.ProcessName=NameNode 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogFatal=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillis=328 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemMaxM=2200.0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTerminated=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsWaiting=87 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.SessionId=null 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Context=jvm 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapUsedM=193.45564 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTotalExtraSleepTime=54 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Old Generation=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCount=11 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsBlocked=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogWarn=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsRunnable=85 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogInfo=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsNew=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapUsedM=564.47656 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTimedWaiting=117 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapMaxM=2200.0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Young Generation=11 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogError=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.ProcessName=DataNode 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogFatal=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillis=328 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_90th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_25th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_num_ops=16 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99.9th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_90th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_95th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_median=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_max=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_95th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_98th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_98th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_25th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_TimeRangeCount_1-3=1 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:fsChecksumFailureCount=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_median=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_75th_percentile=2 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_min=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Context=regionserver 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_max=2 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_min=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_90th_percentile=2 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_75th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_max=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99.9th_percentile=0 2023-10-15T16:24:16,822 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_98th_percentile=2 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_mean=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_95th_percentile=2 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_num_ops=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_min=2 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_median=2 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99.9th_percentile=2 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_mean=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_num_ops=214 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_25th_percentile=2 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_mean=2 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_75th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99th_percentile=2 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context=dfs 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.1=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.2=dfs 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.2=8c5d72df62de 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.1=dfs 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.1=8c5d72df62de 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.2=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Context=master 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:numMasterWALs=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsCount=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_98th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_median=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestRatePerSecond=0.0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerMissCount=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_mean=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_median=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_98th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_95th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_median=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_90th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_5min_rate=0.0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_min=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_95th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitRatio=0.0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_75th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_90th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_num_ops=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_min=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_25th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestCount=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedOutputBytes=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_90th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkHitCount=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadOpsInCurThread=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_median=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestRatePerSecond=0.0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_25th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_min=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_num_ops=0 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_num_ops=2 2023-10-15T16:24:16,823 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_90th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockBytesScannedCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_max=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_75th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_max=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_mean=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_num_ops=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_25th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheEvictionCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99.9th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:smallCompactionQueueLength=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_min=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_num_ops=6 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_mean=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowIncrementCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushQueueLength=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_mean=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_75th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:localBytesRead=123986 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_median=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutateFailedCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_max=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorPoolAllocationBytes=199680 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_98th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_min=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_median=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_min=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:updatesBlockedTime=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_median=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_mean=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitQueueLength=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_min=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_min=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_95th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCachingCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_25th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99.9th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileAge=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_median=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_75th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_max=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_95th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_num_ops=1 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_min=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cpRequestCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_90th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_median=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_max=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_num_ops=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_98th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitRatio=0.0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataMissCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_75th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_mean=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheEvictionCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheFreeSize=922070024 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_95th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_median=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_75th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_num_ops=1 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_max=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_median=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:activeScanners=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissCount=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_mean=0 2023-10-15T16:24:16,824 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_max=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_max=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_95th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactionQueueLength=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_95th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_num_ops=1 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_max=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_1min_rate=0.0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_min=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_median=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_num_ops=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_median=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_mean=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_75th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_min=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99.9th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_median=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_25th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_25th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_90th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_num_ops=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheSize=676856 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_num_ops=2 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_98th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_count=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_median=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowGetCount=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_median=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterRequestsCount=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_min=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_min=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_25th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_max=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileSize=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_min=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileCount=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsCount=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99.9th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_98th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_num_ops=0 2023-10-15T16:24:16,825 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedOutputBytes=5860 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_max=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_num_ops=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_max=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticBloomSize=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexMissCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseInfoThresholdExceeded=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_max=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedToMob=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedMemstoreBytes=928 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheExpressHitPercent=0.0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionServerStartTime=1697385851561 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedInputBytes=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedFromMob=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_max=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRequestCount=11 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_25th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcGetRequestCount=3 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_90th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCountHitPercent=0.0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_max=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_mean=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_min=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_98th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_min=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsSize=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_mean=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowDeleteCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockedRequestCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_mean=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheAccessCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_min=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_median=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_min=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_90th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_min=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_95th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_90th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheEvictedCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_90th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_max=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:avgStoreFileAge=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_25th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitRequestCount=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_min=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalBytesRead=123986 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedToMob=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_max=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedInputBytes=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheSize=676856 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileCount=479 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_mean=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcScanRequestCount=2 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreOffHeapSize=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_75th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99th_percentile=0 2023-10-15T16:24:16,826 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheMissCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowPutCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_max=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreSize=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_75th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsSize=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_mean=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_min=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_75th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_median=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_98th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_median=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Context=regionserver 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:scannerLeaseExpiredCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:minStoreFileAge=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_98th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcFullScanRequestCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_mean=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitSuccessCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_mean=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_75th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_25th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCountPrimary=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_max=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataBlockCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_98th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_max=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_median=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_mean=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCachingCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFailedInsertionCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_90th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsSize=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean_rate=0.0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticIndexSize=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_min=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99.9th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissRatio=0.0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_75th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_max=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_mean=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_95th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMultiRequestCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheSize=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexHitCount=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_max=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_98th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedOutputBytes=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocal=0.0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_98th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_max=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_num_ops=3 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99.9th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_25th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_min=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_mean=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_15min_rate=0.0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99.9th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_min=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:16,827 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_95th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_75th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_median=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_median=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_mean=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_95th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitCount=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:largeCompactionQueueLength=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_mean=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutatePassedCount=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_max=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_95th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissRatio=0.0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsCount=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_min=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_mean=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_min=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:shortCircuitBytesRead=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99.9th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_median=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_min=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoMissCount=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileIndexSize=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99.9th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_mean=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerHitCount=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_98th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsSize=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCountPrimary=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_min=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_max=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99.9th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_90th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_90th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_num_ops=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_max=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_mean=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_max=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_min=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRowActionRequestCount=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMutateRequestCount=6 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheCount=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_25th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_mean=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataHitCount=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:zeroCopyBytesRead=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_98th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionCount=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_num_ops=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileCount=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_num_ops=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_median=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALSize=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_num_ops=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_mean=0 2023-10-15T16:24:16,828 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_median=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheFreeSize=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:averageRegionSize=928 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_95th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaMissCount=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_75th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:filteredReadRequestCount=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaHitCount=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_75th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_num_ops=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_95th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_median=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCount=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_num_ops=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.serverName=8c5d72df62de,34443,1697385851561 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReads=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowAppendCount=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:numReferenceFiles=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedFromMob=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSize=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_max=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_90th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_median=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_mean=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_98th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadWins=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseWarnThresholdExceeded=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_max=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_25th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoHitCount=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_median=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreHeapSize=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_25th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_min=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99.9th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_25th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_num_ops=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_90th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_max=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99.9th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_max=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_median=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsSize=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_median=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_median=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_min=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkMissCount=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,829 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_95th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFreeSize=922070024 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheHitPercent=0.0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsCount=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_98th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_median=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestRatePerSecond=0.0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerMissCount=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_mean=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_median=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_98th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_95th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_median=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_90th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_5min_rate=0.0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_min=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_95th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitRatio=0.0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_75th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_90th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_num_ops=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_min=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_25th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestCount=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedOutputBytes=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_90th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkHitCount=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadOpsInCurThread=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_median=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestRatePerSecond=0.0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_25th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_min=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_num_ops=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_num_ops=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_90th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockBytesScannedCount=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_max=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_75th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_max=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_mean=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_num_ops=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_25th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheEvictionCount=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99.9th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheCount=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:smallCompactionQueueLength=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCount=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_min=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_num_ops=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,830 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_mean=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowIncrementCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushQueueLength=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_mean=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_75th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:localBytesRead=123986 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_median=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutateFailedCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_max=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorPoolAllocationBytes=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_98th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_min=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_median=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_min=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:updatesBlockedTime=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_median=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_mean=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitQueueLength=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_min=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_min=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_95th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCachingCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_25th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99.9th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileAge=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_median=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_75th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_max=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_95th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_num_ops=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_min=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cpRequestCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_90th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_median=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_max=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_num_ops=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_98th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitRatio=0.0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataMissCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_75th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_mean=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheEvictionCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheFreeSize=922070024 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_95th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_median=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_75th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_max=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_median=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:activeScanners=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_mean=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_max=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_max=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_95th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactionQueueLength=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_95th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_num_ops=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_max=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_1min_rate=0.0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_min=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_median=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_num_ops=0 2023-10-15T16:24:16,831 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_median=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_mean=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_75th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_min=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99.9th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_median=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_25th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_25th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_90th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_num_ops=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheSize=676856 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_num_ops=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_98th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_count=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_median=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowGetCount=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_median=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterRequestsCount=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_min=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_min=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_25th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_max=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileSize=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_min=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileCount=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsCount=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99.9th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_98th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_num_ops=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedOutputBytes=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_max=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_num_ops=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_max=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticBloomSize=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexMissCount=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseInfoThresholdExceeded=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCount=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCount=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_max=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedToMob=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedMemstoreBytes=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheExpressHitPercent=0.0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionServerStartTime=1697385851811 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedInputBytes=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedFromMob=0 2023-10-15T16:24:16,832 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_max=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRequestCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_25th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcGetRequestCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_90th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCountHitPercent=0.0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_max=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_mean=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_min=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_98th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_min=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsSize=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_mean=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowDeleteCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockedRequestCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_mean=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheAccessCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_min=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_median=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_min=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_90th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_min=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_95th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_90th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheEvictedCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_90th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_max=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:avgStoreFileAge=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_25th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitRequestCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_min=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalBytesRead=123986 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedToMob=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_max=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedInputBytes=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheSize=676856 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileCount=240 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_mean=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcScanRequestCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreOffHeapSize=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_75th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheMissCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowPutCount=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_max=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreSize=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_75th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsSize=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_mean=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_min=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_75th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_median=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_98th_percentile=0 2023-10-15T16:24:16,833 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_median=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Context=regionserver 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:scannerLeaseExpiredCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:minStoreFileAge=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_98th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcFullScanRequestCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_mean=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitSuccessCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_mean=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_75th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_25th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCountPrimary=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_max=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataBlockCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_98th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_max=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_median=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_mean=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCachingCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFailedInsertionCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_90th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsSize=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean_rate=0.0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticIndexSize=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorUsedBufferCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_min=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99.9th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissRatio=0.0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_75th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_max=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_mean=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_95th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMultiRequestCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheSize=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexHitCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_max=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_98th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedOutputBytes=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocal=0.0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_98th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_max=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_num_ops=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99.9th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_25th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_min=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_mean=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_15min_rate=0.0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99.9th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_min=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_95th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_75th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_median=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_median=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_mean=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_95th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:largeCompactionQueueLength=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_mean=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutatePassedCount=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_max=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_95th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:16,834 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissRatio=0.0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_min=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_mean=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_min=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:shortCircuitBytesRead=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99.9th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_median=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_min=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoMissCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileIndexSize=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99.9th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_mean=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerHitCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_98th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsSize=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCountPrimary=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_min=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_max=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99.9th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_90th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_90th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_num_ops=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_max=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_mean=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_max=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_min=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRowActionRequestCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMutateRequestCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_25th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_mean=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataHitCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:zeroCopyBytesRead=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_98th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_num_ops=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_num_ops=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_median=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALSize=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_num_ops=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_mean=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_median=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheFreeSize=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:averageRegionSize=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_95th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaMissCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_75th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:filteredReadRequestCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaHitCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_75th_percentile=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_num_ops=0 2023-10-15T16:24:16,835 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_95th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_median=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCount=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_num_ops=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.serverName=8c5d72df62de,40585,1697385851811 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReads=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowAppendCount=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:numReferenceFiles=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedFromMob=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSize=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_max=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_90th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_median=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_mean=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_98th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadWins=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseWarnThresholdExceeded=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_max=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_25th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoHitCount=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_median=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreHeapSize=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_25th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_min=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99.9th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_25th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_num_ops=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_90th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_max=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99.9th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_max=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_median=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsSize=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_median=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_median=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_min=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkMissCount=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_95th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFreeSize=922070024 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheHitPercent=0.0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:16,836 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsCount=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_98th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_median=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestRatePerSecond=0.0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerMissCount=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_mean=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_median=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_98th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_95th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_median=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_90th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_5min_rate=0.0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_min=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_95th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitRatio=0.0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_75th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_90th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_num_ops=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_min=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_25th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestCount=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedOutputBytes=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_90th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkHitCount=1 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadOpsInCurThread=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_median=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestRatePerSecond=0.0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_25th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_min=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_num_ops=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_num_ops=11 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_90th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockBytesScannedCount=6840 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_max=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_75th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_max=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_mean=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_num_ops=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_25th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheEvictionCount=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99.9th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheCount=4 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:smallCompactionQueueLength=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCount=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_min=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_num_ops=7 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_num_ops=1 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_mean=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowIncrementCount=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushQueueLength=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_mean=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_75th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:localBytesRead=123986 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushCount=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_median=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutateFailedCount=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_max=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorPoolAllocationBytes=599040 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_98th_percentile=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_min=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_median=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_min=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:updatesBlockedTime=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_median=0 2023-10-15T16:24:16,837 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_mean=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitQueueLength=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_min=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_min=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_95th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCachingCount=4 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_25th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99.9th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileAge=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_median=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_75th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_max=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_95th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_num_ops=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_min=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cpRequestCount=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_90th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_median=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_max=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_num_ops=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALCount=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_98th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitRatio=0.72 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataMissCount=6 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestCount=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissCount=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_75th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_mean=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheEvictionCount=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheFreeSize=922061632 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_95th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_median=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_75th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_max=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_median=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:activeScanners=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissCount=7 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_mean=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_max=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_max=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_95th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactionQueueLength=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_95th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_num_ops=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_max=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_1min_rate=0.0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_min=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_median=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_num_ops=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_median=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_mean=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_75th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_min=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99.9th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_median=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_25th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_25th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_90th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_num_ops=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheSize=685248 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_num_ops=7 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,838 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_num_ops=11 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_98th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_count=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_median=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowGetCount=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_median=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterRequestsCount=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_min=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_min=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_25th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_num_ops=1 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_max=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileSize=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_min=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileCount=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsCount=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99.9th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_98th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_num_ops=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedOutputBytes=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_max=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_num_ops=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_max=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticBloomSize=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexMissCount=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseInfoThresholdExceeded=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCount=4 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCount=7 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_max=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedToMob=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedMemstoreBytes=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheExpressHitPercent=81.81818181818183 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionServerStartTime=1697385888915 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedInputBytes=18105 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedFromMob=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_max=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRequestCount=21 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_25th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcGetRequestCount=1 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_90th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeCount=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCountHitPercent=72.0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_max=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_mean=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_min=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_98th_percentile=0 2023-10-15T16:24:16,839 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_min=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsSize=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_mean=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowDeleteCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockedRequestCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_mean=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheAccessCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_min=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_median=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_min=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_90th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_min=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_95th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_90th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheEvictedCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_90th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_max=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:avgStoreFileAge=12090 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_25th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitRequestCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_num_ops=1 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_min=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalBytesRead=123986 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedToMob=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_max=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedInputBytes=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheSize=685248 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileCount=699 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_mean=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcScanRequestCount=11 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreOffHeapSize=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_75th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheMissCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowPutCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_max=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreSize=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_75th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsSize=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_mean=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_min=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_75th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_median=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_98th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_median=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Context=regionserver 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:scannerLeaseExpiredCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:minStoreFileAge=2959 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_98th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcFullScanRequestCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_mean=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitSuccessCount=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_mean=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_75th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_25th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCountPrimary=7 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_max=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataBlockCount=3 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,840 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_98th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_max=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_median=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsCount=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_mean=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsCount=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCachingCount=18 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFailedInsertionCount=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_90th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsSize=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean_rate=0.0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticIndexSize=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_min=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99.9th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissRatio=0.28 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_75th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_max=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_mean=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_95th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMultiRequestCount=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheSize=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexHitCount=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_max=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_98th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedOutputBytes=5921 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocal=0.0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_98th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_max=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_num_ops=1 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99.9th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_25th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_min=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_mean=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_15min_rate=0.0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99.9th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_min=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_95th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_75th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_median=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_median=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_mean=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_95th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitCount=18 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:largeCompactionQueueLength=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_mean=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutatePassedCount=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_max=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_95th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissRatio=0.0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsCount=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_min=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_mean=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_min=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:shortCircuitBytesRead=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99.9th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_median=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_min=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:16,841 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoMissCount=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileIndexSize=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99.9th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_mean=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerHitCount=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_98th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsSize=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCountPrimary=18 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_min=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_max=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99.9th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_90th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_90th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_num_ops=1 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_max=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_mean=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_max=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_min=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRowActionRequestCount=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMutateRequestCount=8 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheCount=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_25th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationBytes=111031 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_mean=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataHitCount=17 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:zeroCopyBytesRead=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_98th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionCount=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_num_ops=1 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileCount=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_num_ops=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_median=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALSize=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_num_ops=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_mean=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_median=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheFreeSize=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:averageRegionSize=32082 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_95th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaMissCount=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_75th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:filteredReadRequestCount=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaHitCount=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_75th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_num_ops=1 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_95th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_median=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCount=18 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_num_ops=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.serverName=8c5d72df62de,35419,1697385888915 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReads=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowAppendCount=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:numReferenceFiles=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedFromMob=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSize=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_max=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_90th_percentile=0 2023-10-15T16:24:16,842 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_median=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_mean=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_98th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadWins=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseWarnThresholdExceeded=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_max=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_25th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoHitCount=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_median=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreHeapSize=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_25th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_min=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99.9th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_25th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_num_ops=1 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_90th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_max=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99.9th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_max=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_median=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsSize=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_median=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_median=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_min=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkMissCount=1 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_95th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFreeSize=922061632 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheHitPercent=0.0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_98th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_num_ops=2 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.walReaderEditsBufferUsage=80117 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_25th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfHFileRefsQueue=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logReadInBytes=83847 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_75th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfLogQueue=-3 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_90th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99.9th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_num_ops=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedOps=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_min=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedHFiles=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_min=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_mean=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBytes=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_90th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedOps=1000 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedBatches=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_98th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_max=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_95th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedBatches=1 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_mean=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_95th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedHFiles=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_median=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.uncleanlyClosedLogs=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99th_percentile=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.closedLogsWithUnknownFileLength=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_max=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.repeatedLogFileBytes=0 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedRecoverQueues=5 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Context=regionserver 2023-10-15T16:24:16,843 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedRecoverQueues=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.restartedLogReading=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.ignoredUncleanlyClosedLogContentsInBytes=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsRead=40 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.numInitializing=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99.9th_percentile=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_75th_percentile=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsFiltered=38 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_25th_percentile=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedLogs=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.failedBatches=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBatches=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_median=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsAvgTime=0.0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpNumOps=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWrite=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosNumOps=26 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRemoved=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsNumOps=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksCached=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsAvgTime=94.0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalNumOps=9 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesRead=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpAvgTime=6326.928571428572 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesAvgTime=0.0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionTasks=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncCount=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:NumProcessedCommands=10 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpNumOps=27 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromLocalClient=28 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:VolumeFailures=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcInvalidReconstructionTasks=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosNumOps=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesRead=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksReadHits=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosAvgTime=561936.0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalReadTime=9 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesRead=52810 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsAvgTime=1.7142857142857142 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromRemoteClient=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksUncached=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DatanodeNetworkErrors=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromLocalClient=27 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksInPendingIBR=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockVerificationFailures=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosNumOps=54 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesWrite=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosNumOps=54 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.SessionId=null 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsNumOps=1 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesWritten=620489 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcFailedReconstructionTasks=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRead=27 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReplicated=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Context=dfs 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksVerified=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromRemoteClient=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksGetLocalPathInfo=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosAvgTime=20303.65 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpNumOps=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsNumOps=9 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksWritten=28 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsAvgTime=1.0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SumOfActorCommandQueueLength=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalWriteTime=12 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvicted=0 2023-10-15T16:24:16,844 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcDecodingTimeNanos=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesNumOps=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosNumOps=1285 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpNumOps=28 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosAvgTime=23635.714285714286 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesWritten=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosAvgTime=0.0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesWritten=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosAvgTime=10588.8 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpNumOps=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumFailedVolumes=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheCapacity=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheUsed=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:FailedStorageLocations=[Ljava.lang.String;@3cd39472 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksCached=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:LastVolumeFailureDate=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Capacity=1054589059072 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToUncache=10 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:DfsUsed=1039261 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Remaining=941150946725 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToCache=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numRegionSizeReports=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_min=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_num_ops=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_98th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_num_ops=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numNamespaceInQuotaViolation=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_75th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_95th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_90th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_median=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_max=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.namespaceSpaceQuotaOverview=[] 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numTablesInQuotaViolation=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_num_ops=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99.9th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_max=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_98th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_median=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_min=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_max=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Context=master 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99.9th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_95th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_median=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_mean=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_mean=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_max=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_25th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_mean=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_min=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_75th_percentile=0 2023-10-15T16:24:16,845 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_mean=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_90th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_min=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_median=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.tableSpaceQuotaOverview=[] 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numSpaceQuotas=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_25th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_num_ops=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenSubmittedCount=14 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignSubmittedCount=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_num_ops=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99.9th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_median=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_max=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_95th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_95th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_min=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_mean=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_min=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_median=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_min=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveSubmittedCount=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_25th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_mean=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99.9th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_max=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnFS=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerRegions=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_75th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_95th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignSubmittedCount=13 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_75th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_98th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_num_ops=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_90th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignFailedCount=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_num_ops=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_90th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignFailedCount=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_25th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_98th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_98th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveFailedCount=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_mean=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_75th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeFailedCount=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Context=master 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_num_ops=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseSubmittedCount=1 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:holes=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_75th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99.9th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_95th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_95th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeSubmittedCount=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99.9th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_median=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99.9th_percentile=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_mean=0 2023-10-15T16:24:16,846 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCount=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenSubmittedCount=1 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseFailedCount=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_max=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:inconsistentRegions=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_90th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenFailedCount=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnRS=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99.9th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_75th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_75th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_min=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_25th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerOpenRegions=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_95th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_25th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:overlaps=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenFailedCount=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_90th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_98th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitFailedCount=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_max=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_98th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_mean=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_num_ops=13 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_median=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_mean=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCountOverThreshold=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_min=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_75th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_90th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_num_ops=1 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_75th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_95th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_25th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_median=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_25th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitSubmittedCount=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_mean=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritOldestAge=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:emptyRegionInfoRegions=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_max=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_25th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99.9th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_90th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_min=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99.9th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_num_ops=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_max=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_95th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_median=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_max=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_90th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_25th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_25th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_min=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_90th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_95th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_max=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_num_ops=1 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_min=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:operationCount=15 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_98th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:deadServerOpenRegions=3 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_mean=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_median=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_min=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99.9th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_98th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_median=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_max=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_mean=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_num_ops=14 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_98th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_90th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_median=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_98th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_75th_percentile=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumFailedVolumes=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheUsed=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheCapacity=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksCached=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:LastVolumeFailureDate=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Capacity=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Context=FSDatasetState 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data2]'} 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:DfsUsed=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Remaining=0 2023-10-15T16:24:16,847 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToCache=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToUnCache=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalDataFileIos=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalFileIoErrors=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateNumOps=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateNumOps=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateNumOps=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateNumOps=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateNumOps=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateNumOps=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Context=dfs 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalMetadataOperations=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsNumOps=17 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportNumOps=14 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsNumOps=57 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoNumOps=707 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestNumOps=3 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedNumOps=172 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateNumOps=64 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.port=39027 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeNumOps=3 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportAvgTime=1.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Context=rpcdetailed 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameNumOps=44 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportAvgTime=0.21428571428571427 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteAvgTime=0.25 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateAvgTime=5.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteNumOps=64 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingNumOps=219 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsNumOps=334 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportNumOps=4 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeAvgTime=0.6666666666666666 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionNumOps=1 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockNumOps=53 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteNumOps=41 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateNumOps=1 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesNumOps=16 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeNumOps=1 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatNumOps=1200 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseNumOps=39 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseNumOps=5 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatAvgTime=0.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemMaxM=2200.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTerminated=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsWaiting=87 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.SessionId=null 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Context=jvm 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapUsedM=193.45564 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTotalExtraSleepTime=55 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Old Generation=0 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCount=11 2023-10-15T16:24:16,848 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsBlocked=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogWarn=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsRunnable=84 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogInfo=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsNew=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapUsedM=566.47656 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTimedWaiting=118 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapMaxM=2200.0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Young Generation=11 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogError=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.ProcessName=DataNode 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogFatal=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillis=328 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDataNodeMessageCount=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastWrittenTransactionId=466 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:SnapshottableDirectories=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEncryptionZones=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReconstructionBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastCheckpointTime=1697385856136 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ReadLockLongHoldCount=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumActiveClients=1 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastLogRoll=466 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumTimedOutPendingReconstructions=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:VolumeFailuresTotal=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ProvidedCapacityTotal=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingECBlockGroups=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedGB=0.0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecommissioningDataNodes=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.HAState=active 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyECBlockGroups=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:FilesTotal=109 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ScheduledReplicationBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureECBlockGroups=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptECBlockGroups=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MillisSinceLastLoadedEdits=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalECBlockGroups=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExcessBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalLoad=9 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExpiredHeartbeats=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInServiceLiveDataNodes=3 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumStaleStorages=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:WriteLockLongHoldCount=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlocksTotal=43 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:UnderReplicatedBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicatedBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsed=3126472 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalReplicatedBlocks=43 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:StaleDataNodes=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumFilesUnderConstruction=1 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemaining=2823484494063 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PostponedMisreplicatedBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedNonDFS=178715290424 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LockQueueLength=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptReplicatedBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Context=dfs 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomDeadDataNodes=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:Snapshots=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionECBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.TotalSyncTimes=11 5 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReplicationBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotal=3163767177216 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDeadDataNodes=0 2023-10-15T16:24:16,849 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplOneBlocks=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalSyncCount=417 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastCheckpoint=466 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicationOneBlocks=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumLiveDataNodes=3 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlockCapacity=4194304 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotalGB=2946.0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyBlocks=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomLiveDataNodes=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemainingGB=2630.0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:ReceivedBytes=15456 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcSlowCalls=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationSuccesses=46 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumDroppedConnections=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcClientBackoff=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationFailures=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumOpenConnections=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.port=35085 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:SentBytes=9822 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeNumOps=46 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationSuccesses=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:CallQueueLength=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Context=rpc 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationFailures=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeNumOps=46 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeNumOps=46 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:receivedBytes=288613 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationSuccesses=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_max=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFailures=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numOpenConnections=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_num_ops=1195 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_median=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationSuccesses=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:24:16,850 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_num_ops=1195 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_num_ops=1195 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_max=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_max=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_median=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.otherExceptions=5 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Context=regionserver 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_median=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_min=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_median=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_max=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_min=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_min=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveHandler=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_num_ops=1195 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_mean=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReadHandler=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_median=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_num_ops=1195 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_mean=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callDropped=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions=5 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFallbacks=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationFailures=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_min=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_max=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:queueSize=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveScanHandler=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_min=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:sentBytes=44933 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_mean=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_75th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_max=0 2023-10-15T16:24:16,851 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_90th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_min=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_median=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_num_ops=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOnHeapSize=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_num_ops=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_num_ops=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_median=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_num_ops=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_75th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_min=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOffHeapSize=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_25th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_75th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_mean=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_25th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_25th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_mean=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tunerDoNothingCounter=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_25th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_num_ops=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:aboveHeapOccupancyLowWaterMarkCounter=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_98th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreSize=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_min=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_max=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_90th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_mean=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_75th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Context=regionserver 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99.9th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_mean=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_min=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockCacheSize=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_90th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockedFlushGauge=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_max=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_median=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_98th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:unblockedFlushGauge=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_max=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_max=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_num_ops=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_min=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_95th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_90th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_95th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_max=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_median=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_98th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_min=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99.9th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_median=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_median=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_95th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_mean=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_95th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_98th_percentile=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalDataFileIos=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalFileIoErrors=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateNumOps=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateNumOps=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateNumOps=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateNumOps=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateNumOps=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateNumOps=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Context=dfs 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalMetadataOperations=0 2023-10-15T16:24:16,852 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:HttpPort=null 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XceiverCount=3 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"2","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"1199"}] 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:ClusterId=testClusterID 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3":{"numBlocks":21,"usedSpace":259482,"freeSpace":470508168613,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":134216283},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4":{"numBlocks":22,"usedSpace":779779,"freeSpace":470642348032,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0}} 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DatanodeHostname=127.0.0.1 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SecurityEnabled=false 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:RpcPort=0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DataPort=0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SoftwareVersion=3.2.4 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SlowDisks=null 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XmitsInProgress=0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DNStartedTimeInMillis=1697385857021 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsBatchedInSync=50 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsNumOps=416 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsAvgTime=0.0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportNumOps=0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeNumOps=0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteSnapshotOps=0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsAvgTime=0.05714285714285714 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AllowSnapshotOps=0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:RenameSnapshotOps=0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeAvgTime=0.05 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesCreated=176 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesAppended=0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportAvgTime=0.0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageAvgTime=0.0 2023-10-15T16:24:16,853 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeNumOps=246 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumTimesReReplicationNotScheduled=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SafeModeTime=1 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetListingOps=138 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.ProcessName=NameNode 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TotalFileOps=1372 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSymlinkOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AddBlockOps=53 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DisallowSnapshotOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteFileOps=31 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsNumOps=466 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSnapshotOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PendingEditsCount=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ListSnapshottableDirOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockReceivedAndDeletedOps=172 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.SessionId=null 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FsImageLoadTime=23 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateFileOps=64 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Context=dfs 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesRenamed=44 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TimeoutReReplications=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsQueued=1 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesInGetListingOps=154 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FileInfoOps=707 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetLinkTargetOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetAdditionalDatanodeOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsBatched=71 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesDeleted=47 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SuccessfulReReplications=23 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetBlockLocations=334 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesTruncated=1 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SnapshotDiffReportOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportNumOps=8 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportAvgTime=0.5 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumFailedVolumes=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheUsed=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheCapacity=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksCached=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:LastVolumeFailureDate=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Capacity=1054589059072 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Context=FSDatasetState 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:DfsUsed=1039261 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Remaining=941156091301 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToCache=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToUnCache=10 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalDataFileIos=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalFileIoErrors=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Context=dfs 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalMetadataOperations=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateAvgTime=0.0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessNumOps=1 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureNumOps=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailures=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsNumOps=4 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailuresTotal=0 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Context=ugi 2023-10-15T16:24:16,854 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsAvgTime=29.0 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessAvgTime=10.0 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureAvgTime=0.0 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfSnapshottableDirs=0 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheCapacity=0 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheUsed=0 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ProvidedCapacity=0 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DistinctVersionCount=1 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocksWithReplicationFactorOne=0 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentRemaining=89.244385 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:16,855 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentUsed=9.8821176E-5 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:TotalBlocks=43 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NonDfsUsedSpace=178715290424 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Safemode= 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:UpgradeFinalized=true 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:EnteringMaintenanceNodes={} 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentBlockPoolUsed=9.8821176E-5 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:LiveNodes={"127.0.0.1:40557":{"infoAddr":"127.0.0.1:35357","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:40557","lastContact":2,"usedSpace":1039261,"adminState":"In Service","nonDfsUsedSpace":59571348579,"capacity":1054589059072,"numBlocks":42,"version":"3.2.4","used":1039261,"remaining":941161915813,"blockScheduled":1,"blockPoolUsed":1039261,"blockPoolUsedPercent":9.854653E-5,"volfails":0,"lastBlockReport":19},"127.0.0.1:46365":{"infoAddr":"127.0.0.1:35919","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46365","lastContact":2,"usedSpace":1039261,"adminState":"In Service","nonDfsUsedSpace":59571692643,"capacity":1054589059072,"numBlocks":42,"version":"3.2.4","used":1039261,"remaining":941161571749,"blockScheduled":1,"blockPoolUsed":1039261,"blockPoolUsedPercent":9.854653E-5,"volfails":0,"lastBlockReport":8},"127.0.0.1:46399":{"infoAddr":"127.0.0.1:44887","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46399","lastContact":1,"usedSpace":1047950,"adminState":"In Service","nonDfsUsedSpace":59572249202,"capacity":1054589059072,"numBlocks":42,"version":"3.2.4","used":1047950,"remaining":941161006501,"blockScheduled":1,"blockPoolUsed":1047950,"blockPoolUsedPercent":9.937046E-5,"volfails":0,"lastBlockReport":19}} 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CompileInfo=2022-07-12T11:58Z by ubuntu from branch-3.2.4 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolId=BP-1531809096-172.17.0.3-1697385856100 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ClusterId=testClusterID 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DeadNodes={} 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:JournalTransactionInfo={"MostRecentCheckpointTxId":"0","LastAppliedOrWrittenTxId":"466"} 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Free=2823484494063 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolUsedSpace=3126472 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Threads=289 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocks=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirStatuses={"active":{"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":"IMAGE_AND_EDITS","/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":"IMAGE_AND_EDITS"},"failed":{}} 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:SoftwareVersion=3.2.4 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:VerifyECWithTopologyResult={"isSupported":"false","resultMessage":"The number of DataNodes (3) is less than the minimum required number of DataNodes (9) for the erasure coding policies: RS-6-3-1024k"} 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Used=3126472 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Total=3163767177216 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NodeUsage={"nodeUsage":{"min":"0.00%","median":"0.00%","max":"0.00%","stdDev":"0.00%"}} 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NNStartedTimeInMillis=1697385856352 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DecomNodes={} 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CorruptFiles=[] 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirSize={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":737,"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":737} 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameJournalStatus=[{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"},{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"}] 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemMaxM=2200.0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTerminated=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsWaiting=87 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.SessionId=null 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Context=jvm 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapUsedM=193.46402 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapMaxM=-1.0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTotalExtraSleepTime=55 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Hostname=8c5d72df62de 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Old Generation=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCount=11 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsBlocked=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogWarn=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapCommittedM=1000.0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsRunnable=84 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogInfo=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsNew=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapUsedM=567.47656 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTimedWaiting=118 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapMaxM=2200.0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Young Generation=11 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogError=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.ProcessName=DataNode 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogFatal=0 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillis=328 2023-10-15T16:24:16,856 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlocksTotal=43 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TopUserOpCounts={"timestamp":"2023-10-15T16:24:16+0000","windows":[{"ops":[{"opType":"*","totalCount":0,"allUsers":[],"topUsers":[]}],"windowLenMs":60000},{"ops":[{"opType":"*","totalCount":53,"allUsers":[{"user":"jenkins","count":53}],"topUsers":[{"user":"jenkins","count":53}]},{"opType":"getfileinfo","totalCount":31,"allUsers":[{"user":"jenkins","count":31}],"topUsers":[{"user":"jenkins","count":31}]},{"opType":"rename","totalCount":3,"allUsers":[{"user":"jenkins","count":3}],"topUsers":[{"user":"jenkins","count":3}]},{"opType":"mkdirs","totalCount":3,"allUsers":[{"user":"jenkins","count":3}],"topUsers":[{"user":"jenkins","count":3}]},{"opType":"create","totalCount":4,"allUsers":[{"user":"jenkins","count":4}],"topUsers":[{"user":"jenkins","count":4}]},{"opType":"open","totalCount":12,"allUsers":[{"user":"jenkins","count":12}],"topUsers":[{"user":"jenkins","count":12}]}],"windowLenMs":300000},{"ops":[{"opType":"*","totalCount":1499,"allUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":375},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":375},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}]},{"opType":"datanodeReport","totalCount":14,"allUsers":[{"user":"jenkins","count":14}],"topUsers":[{"user":"jenkins","count":14}]},{"opType":"listStatus","totalCount":219,"allUsers":[{"user":"jenkins","count":79},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}],"topUsers":[{"user":"jenkins","count":79},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}]},{"opType":"safemode_get","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"delete","totalCount":41,"allUsers":[{"user":"jenkins","count":20},{"user":"jenkins.hfs.4","count":13},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}],"topUsers":[{"user":"jenkins","count":20},{"user":"jenkins.hfs.4","count":13},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}]},{"opType":"getfileinfo","totalCount":707,"allUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":168},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}],"topUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":168},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}]},{"opType":"truncate","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"rename","totalCount":44,"allUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":15},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":15},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}]},{"opType":"mkdirs","totalCount":57,"allUsers":[{"user":"jenkins","count":35},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}],"topUsers":[{"user":"jenkins","count":35},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}]},{"opType":"create","totalCount":64,"allUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":21},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":21},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}]},{"opType":"setPermission","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"setTimes","totalCount":16,"allUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}]},{"opType":"open","totalCount":334,"allUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":16}],"topUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":16}]}],"windowLenMs":1500000}]} 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:SnapshotStats={"SnapshottableDirectories":0,"Snapshots":0} 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:UnderReplicatedBlocks=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityUsed=3126472 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEncryptionZones=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityRemaining=2823484494063 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReconstructionBlocks=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomDeadDataNodes=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FSState=Operational 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:VolumeFailuresTotal=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ProvidedCapacityTotal=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecommissioningDataNodes=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReplicationBlocks=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncTimes=11 5 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingDeletionBlocks=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityTotal=3163767177216 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FilesTotal=109 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ScheduledReplicationBlocks=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDeadDataNodes=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncCount=417 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FsLockQueueLength=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:MaxObjects=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalLoad=9 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleDataNodes=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumLiveDataNodes=3 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlockDeletionStartTime=1697385856352 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInServiceLiveDataNodes=3 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:LowRedundancyBlocks=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomLiveDataNodes=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleStorages=0 2023-10-15T16:24:16,858 INFO [pool-249-thread-1 {}] common.MetricsLoggerTask(109): << End DataNode metrics dump 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(83): >> Begin DataNode metrics dump 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_num_ops=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_mean=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_num_ops=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_25th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_95th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_90th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_90th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_max=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_98th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_25th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_75th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_min=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_75th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_min=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_max=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_max=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_25th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99.9th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_75th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_mean=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99.9th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_75th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_median=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_95th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_98th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99.9th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_median=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_25th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Context=master 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_median=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_95th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_98th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_90th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_mean=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99.9th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_95th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_min=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_max=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_98th_percentile=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_mean=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_num_ops=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_min=0 2023-10-15T16:24:17,108 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_median=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_num_ops=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_90th_percentile=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:HttpPort=null 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XceiverCount=3 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"0","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"384","LastBlockReport":"486"}] 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:ClusterId=testClusterID 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1":{"numBlocks":21,"usedSpace":259482,"freeSpace":470507779493,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":134216283},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2":{"numBlocks":22,"usedSpace":779779,"freeSpace":470641995776,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0}} 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DatanodeHostname=127.0.0.1 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SecurityEnabled=false 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:RpcPort=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DataPort=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SoftwareVersion=3.2.4 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SlowDisks=null 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XmitsInProgress=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DNStartedTimeInMillis=1697385856691 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Context=regionserver 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.port=44717 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Context=rpcdetailed 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthNumOps=50 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpNumOps=5 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWrite=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosNumOps=48 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRemoved=10 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalAvgTime=0.8367346938775511 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsNumOps=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksCached=0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsAvgTime=2.0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DataNodeActiveXceiversCount=1 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalNumOps=400 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsNumOps=58 2023-10-15T16:24:17,109 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesRead=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpAvgTime=50.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpAvgTime=225065.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesAvgTime=0.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionTasks=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncCount=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:NumProcessedCommands=401 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpNumOps=104 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromLocalClient=52 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:VolumeFailures=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosNumOps=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesRead=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosAvgTime=818128.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalReadTime=18 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesRead=362275 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsAvgTime=1.4 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromRemoteClient=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksUncached=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DatanodeNetworkErrors=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromLocalClient=104 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksInPendingIBR=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockVerificationFailures=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosNumOps=208 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesWrite=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosNumOps=208 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.SessionId=null 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsNumOps=1 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesWritten=1008141 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRead=104 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReplicated=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Context=dfs 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksVerified=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromRemoteClient=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosAvgTime=26277.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsNumOps=400 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksWritten=52 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsAvgTime=0.7857142857142857 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalWriteTime=21 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcDecodingTimeNanos=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesNumOps=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosNumOps=1724 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpNumOps=52 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosAvgTime=5329.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesWritten=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosAvgTime=10111.5 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpAvgTime=0.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpNumOps=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumFailedVolumes=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheUsed=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheCapacity=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksCached=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:LastVolumeFailureDate=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Capacity=1054589059072 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Context=FSDatasetState 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:DfsUsed=1039261 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Remaining=941156091301 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToCache=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToUnCache=10 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheCleared=1 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheHit=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Context=rpc 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheUpdated=0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,110 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalDataFileIos=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalFileIoErrors=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateNumOps=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateNumOps=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateNumOps=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateNumOps=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateNumOps=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateNumOps=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Context=dfs 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalMetadataOperations=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:MissingECBlockGroups=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:LowRedundancyECBlockGroups=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:BytesInFutureECBlockGroups=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:CorruptECBlockGroups=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:PendingDeletionECBlocks=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:TotalECBlockGroups=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheCleared=1 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheHit=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Context=rpc 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheUpdated=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:LastHATransitionTime=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:HostAndPort=localhost:39027 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:State=active 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowPeersReport=null 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowDisksReport=null 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:NNRole=NameNode 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:BytesWithFutureGenerationStamps=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SecurityEnabled=false 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.port=35753 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Context=rpcdetailed 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthNumOps=55 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:ReceivedBytes=18480 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcSlowCalls=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationSuccesses=55 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumDroppedConnections=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcClientBackoff=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationFailures=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumOpenConnections=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.port=35753 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:SentBytes=11751 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeNumOps=55 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationSuccesses=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:CallQueueLength=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Context=rpc 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationFailures=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeNumOps=55 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeNumOps=55 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:ReceivedBytes=16800 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcSlowCalls=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationSuccesses=50 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumDroppedConnections=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcClientBackoff=0 2023-10-15T16:24:17,111 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationFailures=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumOpenConnections=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.port=44717 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:SentBytes=10673 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeNumOps=50 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationSuccesses=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:CallQueueLength=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Context=rpc 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationFailures=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeNumOps=50 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeNumOps=50 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemMaxM=2200.0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTerminated=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsWaiting=87 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.SessionId=null 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Context=jvm 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapUsedM=193.44197 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTotalExtraSleepTime=51 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Old Generation=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCount=11 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsBlocked=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogWarn=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsRunnable=84 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogInfo=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsNew=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapUsedM=562.47656 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTimedWaiting=118 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapMaxM=2200.0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Young Generation=11 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogError=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.ProcessName=NameNode 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogFatal=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillis=328 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumFailedVolumes=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheUsed=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheCapacity=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksCached=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:LastVolumeFailureDate=0 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Capacity=1054589059072 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Context=FSDatasetState 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:DfsUsed=1047950 2023-10-15T16:24:17,112 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Remaining=941156091301 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToCache=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToUnCache=10 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemMaxM=2200.0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTerminated=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsWaiting=87 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.SessionId=null 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Context=jvm 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapUsedM=193.44412 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTotalExtraSleepTime=22 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Old Generation=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCount=11 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsBlocked=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogWarn=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsRunnable=84 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogInfo=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsNew=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapUsedM=562.47656 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTimedWaiting=118 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapMaxM=2200.0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Young Generation=11 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogError=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.ProcessName=DataNode 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogFatal=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillis=328 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_num_ops=1906 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99th_percentile=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_max=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_mean=187 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:appendCount=1906 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:lowReplicaRollRequest=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99th_percentile=336 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99.9th_percentile=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:sizeRollRequest=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendSize=301542 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_max=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowAppendCount=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:successfulLogRolls=25 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99.9th_percentile=337 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_TimeRangeCount_0-1=4 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_25th_percentile=37 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendSize=9557 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_90th_percentile=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendCount=1011 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_25th_percentile=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_10-100=2 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99th_percentile=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_98th_percentile=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_75th_percentile=297 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_75th_percentile=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_max=338 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_95th_percentile=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_mean=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendCount=48 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_100-1000=2 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:errorRollRequest=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:rollRequest=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_num_ops=1819 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_median=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_98th_percentile=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_min=36 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_min=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_median=0 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Context=regionserver 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_0-1=4 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendCount=45 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_95th_percentile=329 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendSize=74104 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_median=38 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_98th_percentile=334 2023-10-15T16:24:17,113 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_25th_percentile=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_75th_percentile=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_mean=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99.9th_percentile=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendSize=14494 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:writtenBytes=399697 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_90th_percentile=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_95th_percentile=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowSyncRollRequest=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_num_ops=1906 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_90th_percentile=321 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_min=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendCount=802 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicationOneBlocks=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:CorruptReplicatedBlocks=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicatedBlocks=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:TotalReplicatedBlocks=43 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalDataFileIos=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalFileIoErrors=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Context=dfs 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalMetadataOperations=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Context=metricssystem 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSources=67 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:DroppedPubAll=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSources=67 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSinks=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSinks=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheCleared=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheHit=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Context=rpc 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheUpdated=149 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWrite=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosNumOps=28 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRemoved=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsNumOps=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksCached=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsAvgTime=3.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalNumOps=9 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesRead=0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpAvgTime=9.0 2023-10-15T16:24:17,114 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpAvgTime=6810.7692307692305 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesAvgTime=0.0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionTasks=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncCount=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:NumProcessedCommands=10 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpNumOps=31 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromLocalClient=28 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:VolumeFailures=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosNumOps=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesRead=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosAvgTime=554484.75 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalReadTime=7 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesRead=57958 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsAvgTime=1.5714285714285714 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromRemoteClient=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksUncached=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DatanodeNetworkErrors=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromLocalClient=31 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksInPendingIBR=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockVerificationFailures=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosNumOps=62 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesWrite=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosNumOps=62 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.SessionId=null 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsNumOps=1 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesWritten=620489 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRead=31 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReplicated=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Context=dfs 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksVerified=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromRemoteClient=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosAvgTime=19658.18181818182 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsNumOps=9 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksWritten=28 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsAvgTime=1.0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalWriteTime=12 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcDecodingTimeNanos=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesNumOps=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosNumOps=1285 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpNumOps=28 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosAvgTime=4301.205882352941 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesWritten=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosAvgTime=9971.454545454546 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpAvgTime=0.8181818181818182 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpNumOps=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemMaxM=2200.0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTerminated=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsWaiting=87 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.SessionId=null 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Context=jvm 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapUsedM=193.44412 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTotalExtraSleepTime=62 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Old Generation=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCount=11 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsBlocked=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogWarn=0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,115 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsRunnable=84 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogInfo=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsNew=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapUsedM=562.47656 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTimedWaiting=118 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapMaxM=2200.0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Young Generation=11 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogError=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.ProcessName=DataNode 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogFatal=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillis=328 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Context=regionserver 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:numTables=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99.9th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_90th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_95th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterRequests=1101 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterWriteRequests=1059 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_mean=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterReadRequests=74 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_min=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_25th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashSubmittedCount=4 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashFailedCount=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_98th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Context=master 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_num_ops=2 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_75th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_max=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_median=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:receivedBytes=6051329 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_98th_percentile=1 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationSuccesses=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_25th_percentile=16 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_max=1 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_98th_percentile=16 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_90th_percentile=1 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFailures=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_98th_percentile=1 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numOpenConnections=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_75th_percentile=16 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_num_ops=1334 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:24:17,116 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99.9th_percentile=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_median=36 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_90th_percentile=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationSuccesses=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99.9th_percentile=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_max=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_num_ops=1334 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_num_ops=1334 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_max=16 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_max=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_median=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.otherExceptions=3 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_95th_percentile=16 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Context=master 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99th_percentile=16 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_median=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_min=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99.9th_percentile=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_median=16 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_max=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_95th_percentile=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_min=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_98th_percentile=36 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_min=16 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_SizeRangeCount_10-100=30 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99.9th_percentile=16 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99th_percentile=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveHandler=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_num_ops=1334 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_mean=16 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_SizeRangeCount_10-100=30 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReadHandler=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_median=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_90th_percentile=36 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_num_ops=1334 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99th_percentile=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_mean=36 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_95th_percentile=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_25th_percentile=36 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callDropped=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_75th_percentile=36 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions=3 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFallbacks=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99th_percentile=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationFailures=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_min=36 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_95th_percentile=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_max=36 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:queueSize=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99.9th_percentile=36 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveScanHandler=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_95th_percentile=36 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_98th_percentile=1 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_90th_percentile=16 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99th_percentile=36 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_min=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:24:17,117 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:sentBytes=18267 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumFailedVolumes=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheUsed=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheCapacity=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksCached=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:LastVolumeFailureDate=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Capacity=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Context=FSDatasetState 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data4]'} 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:DfsUsed=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Remaining=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToCache=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToUnCache=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:HttpPort=null 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XceiverCount=3 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"1","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"1198"}] 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:ClusterId=testClusterID 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5":{"numBlocks":22,"usedSpace":274432,"freeSpace":470507648421,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":134216283},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6":{"numBlocks":21,"usedSpace":773518,"freeSpace":470641864704,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0}} 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DatanodeHostname=127.0.0.1 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SecurityEnabled=false 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:RpcPort=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DataPort=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SoftwareVersion=3.2.4 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SlowDisks=null 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XmitsInProgress=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DNStartedTimeInMillis=1697385857417 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalDataFileIos=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalFileIoErrors=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateNumOps=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateNumOps=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateNumOps=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateNumOps=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateNumOps=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateNumOps=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Context=dfs 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalMetadataOperations=0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,118 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumFailedVolumes=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheUsed=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheCapacity=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksCached=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:LastVolumeFailureDate=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Capacity=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Context=FSDatasetState 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data6]'} 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:DfsUsed=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Remaining=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToCache=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToUnCache=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:ReceivedBytes=1072216 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcSlowCalls=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationSuccesses=47 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumDroppedConnections=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.NumOpenConnectionsPerUser={"jenkins":1} 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcClientBackoff=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationFailures=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumOpenConnections=1 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.port=39027 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:SentBytes=679332 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeNumOps=3059 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationSuccesses=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:CallQueueLength=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Context=rpc 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationFailures=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeNumOps=3059 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.serverName=ClientNamenodeProtocol 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeNumOps=3059 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.port=35085 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Context=rpcdetailed 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthNumOps=46 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemMaxM=2200.0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTerminated=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsWaiting=87 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.SessionId=null 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Context=jvm 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapUsedM=193.44489 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTotalExtraSleepTime=45 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Old Generation=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCount=11 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsBlocked=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogWarn=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsRunnable=84 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogInfo=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsNew=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapUsedM=562.47656 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTimedWaiting=118 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapMaxM=2200.0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Young Generation=11 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogError=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.ProcessName=DataNode 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogFatal=0 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillis=328 2023-10-15T16:24:17,119 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsBatchedInSync=31 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsNumOps=224 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsAvgTime=0.13043478260869565 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportNumOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeNumOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteSnapshotOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsAvgTime=0.03896103896103896 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AllowSnapshotOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:RenameSnapshotOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeAvgTime=0.0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesCreated=80 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesAppended=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportAvgTime=0.0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageAvgTime=0.0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeNumOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeNumOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeNumOps=3 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeAvgTime=0.0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumTimesReReplicationNotScheduled=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SafeModeTime=3 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalNumOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetListingOps=66 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.ProcessName=NameNode 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageNumOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TotalFileOps=540 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSymlinkOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AddBlockOps=28 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DisallowSnapshotOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteFileOps=5 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsNumOps=255 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSnapshotOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PendingEditsCount=1 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ListSnapshottableDirOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockReceivedAndDeletedOps=96 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.SessionId=null 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditNumOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FsImageLoadTime=173 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateFileOps=36 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Context=dfs 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesRenamed=17 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TimeoutReReplications=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsQueued=1 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedNumOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageNumOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageAvgTime=0.0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesInGetListingOps=35 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FileInfoOps=270 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetLinkTargetOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetAdditionalDatanodeOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsBatched=9 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesDeleted=1 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SuccessfulReReplications=2 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetBlockLocations=118 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesTruncated=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SnapshotDiffReportOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportNumOps=6 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeNumOps=0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditAvgTime=0.0 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportAvgTime=0.5 2023-10-15T16:24:17,120 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumFailedVolumes=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheCapacity=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheUsed=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:FailedStorageLocations=[Ljava.lang.String;@524c11fd 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksCached=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:LastVolumeFailureDate=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Capacity=1054589059072 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToUncache=10 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:DfsUsed=1039261 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Remaining=941149250981 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToCache=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpNumOps=6 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWrite=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosNumOps=54 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRemoved=10 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalAvgTime=0.7474747474747475 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsNumOps=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksCached=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsAvgTime=2.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DataNodeActiveXceiversCount=1 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalNumOps=400 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsNumOps=55 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesRead=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpAvgTime=34.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpAvgTime=225066.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesAvgTime=0.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionTasks=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncCount=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:NumProcessedCommands=401 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpNumOps=99 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromLocalClient=52 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:VolumeFailures=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosNumOps=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesRead=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosAvgTime=924152.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalReadTime=29 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesRead=688752 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsAvgTime=2.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromRemoteClient=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksUncached=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DatanodeNetworkErrors=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromLocalClient=99 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksInPendingIBR=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockVerificationFailures=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosNumOps=200 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesWrite=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosNumOps=200 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.SessionId=null 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsNumOps=1 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesWritten=1008141 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRead=99 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReplicated=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Context=dfs 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksVerified=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromRemoteClient=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosAvgTime=45642.0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsNumOps=400 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksWritten=52 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsAvgTime=0.696969696969697 2023-10-15T16:24:17,121 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalWriteTime=22 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcDecodingTimeNanos=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesNumOps=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosNumOps=1724 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpNumOps=52 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosAvgTime=6378.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesWritten=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosAvgTime=9294.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpNumOps=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWrite=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosNumOps=30 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRemoved=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsNumOps=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksCached=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsAvgTime=95.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalNumOps=9 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesRead=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpAvgTime=6809.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesAvgTime=0.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionTasks=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncCount=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:NumProcessedCommands=10 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpNumOps=35 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromLocalClient=28 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:VolumeFailures=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosNumOps=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesRead=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosAvgTime=352052.375 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalReadTime=16 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesRead=75811 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsAvgTime=1.3846153846153846 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromRemoteClient=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksUncached=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DatanodeNetworkErrors=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromLocalClient=35 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksInPendingIBR=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockVerificationFailures=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosNumOps=70 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesWrite=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosNumOps=70 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.SessionId=null 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsNumOps=1 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesWritten=620489 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRead=35 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReplicated=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Context=dfs 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksVerified=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromRemoteClient=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosAvgTime=25843.916666666668 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsNumOps=9 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksWritten=28 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsAvgTime=1.0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalWriteTime=12 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcDecodingTimeNanos=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesNumOps=0 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosNumOps=1285 2023-10-15T16:24:17,122 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpNumOps=28 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosAvgTime=4740.882352941177 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesWritten=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosAvgTime=9895.041666666666 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpAvgTime=0.75 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpNumOps=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpNumOps=6 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWrite=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosNumOps=34 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRemoved=10 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalAvgTime=0.8877551020408163 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsNumOps=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksCached=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsAvgTime=3.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DataNodeActiveXceiversCount=1 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalNumOps=400 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsNumOps=59 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesRead=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpAvgTime=10.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpAvgTime=225063.25 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesAvgTime=0.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionTasks=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncCount=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:NumProcessedCommands=402 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpNumOps=91 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromLocalClient=52 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:VolumeFailures=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosNumOps=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesRead=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosAvgTime=1214340.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalReadTime=12 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesRead=349472 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsAvgTime=1.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromRemoteClient=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksUncached=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DatanodeNetworkErrors=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromLocalClient=91 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksInPendingIBR=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockVerificationFailures=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosNumOps=182 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesWrite=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosNumOps=182 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.SessionId=null 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsNumOps=2 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,123 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesWritten=1008141 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRead=91 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReplicated=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Context=dfs 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksVerified=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromRemoteClient=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosAvgTime=53154.75 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsNumOps=400 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksWritten=52 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsAvgTime=0.8571428571428571 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalWriteTime=28 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcDecodingTimeNanos=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesNumOps=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosNumOps=1724 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpNumOps=52 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosAvgTime=5041.5 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesWritten=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosAvgTime=8707.0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpNumOps=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDataNodeMessageCount=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastWrittenTransactionId=255 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:SnapshottableDirectories=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEncryptionZones=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReconstructionBlocks=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptBlocks=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastCheckpointTime=1697385845476 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ReadLockLongHoldCount=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumActiveClients=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastLogRoll=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumTimedOutPendingReconstructions=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:VolumeFailuresTotal=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ProvidedCapacityTotal=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingECBlockGroups=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedGB=0.0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecommissioningDataNodes=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.HAState=active 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionBlocks=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyECBlockGroups=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:FilesTotal=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ScheduledReplicationBlocks=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureECBlockGroups=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptECBlockGroups=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MillisSinceLastLoadedEdits=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalECBlockGroups=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExcessBlocks=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalLoad=32 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExpiredHeartbeats=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInServiceLiveDataNodes=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumStaleStorages=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:WriteLockLongHoldCount=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlocksTotal=0 2023-10-15T16:24:17,124 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:UnderReplicatedBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicatedBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsed=495713 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalReplicatedBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:StaleDataNodes=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumFilesUnderConstruction=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemaining=2816043276636 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PostponedMisreplicatedBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedNonDFS=185085906847 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LockQueueLength=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptReplicatedBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Context=dfs 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomDeadDataNodes=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:Snapshots=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionECBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.TotalSyncTimes= 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReplicationBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotal=3163767177216 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDeadDataNodes=3 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplOneBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalSyncCount=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastCheckpoint=255 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicationOneBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumLiveDataNodes=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlockCapacity=4194304 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotalGB=2946.0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyBlocks=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomLiveDataNodes=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemainingGB=2623.0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Context=regionserver 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheCleared=1 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheHit=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Context=rpc 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheUpdated=65 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemMaxM=2200.0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTerminated=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsWaiting=87 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.SessionId= 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Context=jvm 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapUsedM=193.45454 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Old Generation=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCount=11 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsBlocked=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogWarn=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsRunnable=84 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogInfo=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsNew=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapUsedM=563.47656 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTimedWaiting=118 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapMaxM=2200.0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Young Generation=11 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogError=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.ProcessName=Master 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogFatal=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillis=328 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsCount=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeTotal=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointPercentComplete=1.0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageTotal=1 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsTotal=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointCount=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:ElapsedTime=681 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsPercentComplete=1.0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsElapsedTime=7 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointElapsedTime=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModePercentComplete=1.0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:PercentComplete=1.0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointTotal=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeCount=0 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeElapsedTime=13 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageElapsedTime=62 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageCount=1 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,125 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImagePercentComplete=1.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumFailedVolumes=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheCapacity=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheUsed=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:FailedStorageLocations=[Ljava.lang.String;@7860eb42 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksCached=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:LastVolumeFailureDate=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Capacity=1054589059072 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToUncache=10 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:DfsUsed=1047950 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Remaining=941149250981 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToCache=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsCount=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeTotal=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointPercentComplete=1.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageTotal=1 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsTotal=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointCount=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:ElapsedTime=681 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsPercentComplete=1.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsElapsedTime=7 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointElapsedTime=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModePercentComplete=1.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:PercentComplete=1.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointTotal=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeCount=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeElapsedTime=13 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageElapsedTime=62 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageCount=1 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImagePercentComplete=1.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RackLocalityCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionCountSkewCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaHostCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ServerLocalityCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MemStoreSizeCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_95th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_Overall=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ReadRequestCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_25th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_98th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_min=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Context=master 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_WriteRequestCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaRackCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_PrimaryRegionCountSkewCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:miscInvocationCount=18 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_median=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_num_ops=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_75th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MoveCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_max=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_mean=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_CPRequestCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_TableSkewCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_StoreFileCostFunction=0.0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_90th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.isBalancerActive=true 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99.9th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_95th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99.9th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_median=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_mean=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_25th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_max=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_95th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_num_ops=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,126 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_25th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_98th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_min=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_mean=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_90th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_75th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_90th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_mean=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_num_ops=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_95th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_min=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_median=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_num_ops=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_mean=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_98th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_75th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_median=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99.9th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Context=regionserver 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_max=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_num_ops=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_num_ops=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_min=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_90th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_mean=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_95th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_max=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_75th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_75th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_min=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_num_ops=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_75th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_max=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_min=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_98th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_90th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_max=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_98th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_90th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_75th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_95th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_25th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_max=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_max=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_98th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_num_ops=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99.9th_percentile=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_min=0 2023-10-15T16:24:17,127 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_98th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_95th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_min=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_median=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:BlockBytesScannedCount=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99.9th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_25th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_median=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_90th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_95th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_num_ops=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_max=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_mean=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_max=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_min=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_mean=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_95th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99.9th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_25th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99.9th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_90th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_mean=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_95th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_min=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_75th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_max=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_max=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_90th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_max=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_98th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_25th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_median=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_median=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_75th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_mean=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99.9th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_median=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_90th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_mean=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_mean=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_75th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_num_ops=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99.9th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_median=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_75th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_median=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_median=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_98th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_95th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_75th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_95th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_num_ops=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_max=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_98th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_25th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_95th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99.9th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_num_ops=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_25th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99.9th_percentile=0 2023-10-15T16:24:17,128 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_median=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_90th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_25th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_median=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_min=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_max=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_90th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_max=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_75th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_25th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_98th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_min=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_mean=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99.9th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_25th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_min=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_mean=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_min=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_98th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_min=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_98th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99.9th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_median=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_num_ops=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_median=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_90th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_min=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_25th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsCount=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_98th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_median=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestRatePerSecond=0.0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerMissCount=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_mean=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_median=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_98th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_95th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_median=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_90th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_5min_rate=0.0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_min=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_95th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitRatio=0.0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_75th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_90th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,129 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_num_ops=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_min=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_25th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestCount=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedOutputBytes=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_90th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkHitCount=1 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadOpsInCurThread=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_median=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestRatePerSecond=0.0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_25th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_min=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_num_ops=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_num_ops=26 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_90th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockBytesScannedCount=4389 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_max=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_75th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_max=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_mean=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_num_ops=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_25th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheEvictionCount=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99.9th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheCount=3 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:smallCompactionQueueLength=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCount=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_min=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_num_ops=15 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_mean=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowIncrementCount=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushQueueLength=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_mean=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_75th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:localBytesRead=123986 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushCount=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_median=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutateFailedCount=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_max=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorPoolAllocationBytes=1530880 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_98th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_min=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_median=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_min=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:updatesBlockedTime=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_median=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_mean=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitQueueLength=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_min=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_min=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_95th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCachingCount=3 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_25th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99.9th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileAge=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_median=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_75th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_max=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_95th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_num_ops=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_min=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cpRequestCount=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_90th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_median=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_max=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99th_percentile=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_num_ops=0 2023-10-15T16:24:17,130 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALCount=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_98th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitRatio=0.7272727272727273 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataMissCount=1 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestCount=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissCount=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_75th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_mean=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheEvictionCount=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheFreeSize=922067336 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_95th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_median=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_75th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_max=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_median=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:activeScanners=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissCount=3 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_mean=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_max=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_max=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_95th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactionQueueLength=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_95th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_num_ops=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_max=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_1min_rate=0.0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_min=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_median=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_num_ops=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_median=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_mean=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_75th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_min=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99.9th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_median=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_25th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_25th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_90th_percentile=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_num_ops=0 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheSize=679544 2023-10-15T16:24:17,131 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_num_ops=6 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_num_ops=26 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_98th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_count=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_median=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowGetCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_median=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterRequestsCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_min=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_min=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_25th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_max=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileSize=836094 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_min=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99.9th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_98th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_num_ops=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedOutputBytes=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_max=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_num_ops=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_max=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticBloomSize=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexMissCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseInfoThresholdExceeded=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCount=3 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCount=3 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_max=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedToMob=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedMemstoreBytes=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheExpressHitPercent=72.72727272727273 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionServerStartTime=1697385859285 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedInputBytes=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedFromMob=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_max=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRequestCount=47 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_25th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcGetRequestCount=3 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_90th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCountHitPercent=72.72727272727273 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_max=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_mean=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_min=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_98th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_min=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsSize=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_mean=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowDeleteCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockedRequestCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_mean=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheAccessCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_min=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_median=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_min=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_90th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_min=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_95th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_90th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheEvictedCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_90th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_max=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:avgStoreFileAge=1819 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_25th_percentile=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitRequestCount=0 2023-10-15T16:24:17,132 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_num_ops=1 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_min=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalBytesRead=123986 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedToMob=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_max=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedInputBytes=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheSize=679544 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileCount=953 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_mean=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcScanRequestCount=26 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreOffHeapSize=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_75th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheMissCount=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowPutCount=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_max=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreSize=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_75th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsSize=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_mean=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_min=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_75th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_median=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_98th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_median=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Context=regionserver 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:scannerLeaseExpiredCount=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:minStoreFileAge=1782 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitCount=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_98th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcFullScanRequestCount=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_mean=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitSuccessCount=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_mean=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_75th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_25th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCountPrimary=3 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_max=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataBlockCount=1 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_98th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_max=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_median=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsCount=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_mean=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsCount=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCachingCount=8 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFailedInsertionCount=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_90th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsSize=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean_rate=0.0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticIndexSize=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorUsedBufferCount=2 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_min=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99.9th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissRatio=0.2727272727272727 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_75th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_max=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_mean=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_95th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMultiRequestCount=1 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheSize=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexHitCount=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_max=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_98th_percentile=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedOutputBytes=0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocal=0.0 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:17,133 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_98th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_max=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_num_ops=3 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99.9th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_25th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_min=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_mean=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_15min_rate=0.0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99.9th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_min=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_95th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_75th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_median=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_median=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_mean=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_95th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitCount=8 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:largeCompactionQueueLength=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_mean=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutatePassedCount=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_max=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_95th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissRatio=0.0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsCount=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_min=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_mean=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_min=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:shortCircuitBytesRead=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99.9th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_median=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_min=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoMissCount=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,134 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileIndexSize=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99.9th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_mean=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerHitCount=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_98th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsSize=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCountPrimary=8 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_min=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_max=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99.9th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_90th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_90th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_num_ops=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_max=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_mean=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_max=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_min=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRowActionRequestCount=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMutateRequestCount=16 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheCount=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_25th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_mean=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataHitCount=7 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:zeroCopyBytesRead=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_98th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionCount=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_num_ops=1 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileCount=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_num_ops=1 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_median=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALSize=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_num_ops=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_mean=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_median=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheFreeSize=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:averageRegionSize=17474 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_95th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaMissCount=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_75th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:filteredReadRequestCount=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaHitCount=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_75th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_num_ops=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_95th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_median=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCount=8 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_num_ops=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.serverName=8c5d72df62de,45999,1697385859285 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReads=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowAppendCount=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:numReferenceFiles=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedFromMob=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSize=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_max=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationRatio=1.0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_90th_percentile=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_median=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_mean=0 2023-10-15T16:24:17,135 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_98th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadWins=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseWarnThresholdExceeded=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_max=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_25th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoHitCount=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_median=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreHeapSize=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_25th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_min=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99.9th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_25th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_num_ops=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_90th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_max=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99.9th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_max=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_median=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsSize=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_median=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_median=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_min=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkMissCount=2 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_95th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFreeSize=922067336 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheHitPercent=0.0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemMaxM=2200.0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTerminated=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsWaiting=87 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.SessionId=null 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Context=jvm 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapUsedM=193.45564 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTotalExtraSleepTime=46 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Old Generation=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCount=11 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsBlocked=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogWarn=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsRunnable=84 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogInfo=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsNew=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapUsedM=564.47656 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTimedWaiting=118 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapMaxM=2200.0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Young Generation=11 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogError=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.ProcessName=NameNode 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogFatal=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillis=328 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemMaxM=2200.0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTerminated=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsWaiting=87 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.SessionId=null 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Context=jvm 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapUsedM=193.45564 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTotalExtraSleepTime=54 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Old Generation=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCount=11 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsBlocked=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogWarn=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsRunnable=85 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogInfo=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsNew=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapUsedM=564.47656 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTimedWaiting=117 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapMaxM=2200.0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Young Generation=11 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogError=0 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.ProcessName=DataNode 2023-10-15T16:24:17,136 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogFatal=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillis=328 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_90th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_25th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_num_ops=16 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99.9th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_90th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_95th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_median=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_max=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_95th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_98th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_98th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_25th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_TimeRangeCount_1-3=1 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:fsChecksumFailureCount=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_median=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_75th_percentile=2 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_min=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Context=regionserver 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_max=2 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_min=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_90th_percentile=2 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_75th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_max=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99.9th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_98th_percentile=2 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_mean=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_95th_percentile=2 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_num_ops=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_min=2 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_median=2 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99.9th_percentile=2 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_mean=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_num_ops=214 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_25th_percentile=2 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_mean=2 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_75th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99th_percentile=2 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context=dfs 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.1=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.2=dfs 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.2=8c5d72df62de 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.1=dfs 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.1=8c5d72df62de 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.2=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Context=master 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:numMasterWALs=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsCount=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_98th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_median=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestRatePerSecond=0.0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerMissCount=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_mean=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_median=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_98th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_95th_percentile=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_median=0 2023-10-15T16:24:17,137 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_90th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_5min_rate=0.0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_min=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_95th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitRatio=0.0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_75th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_90th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_num_ops=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_min=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_25th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestCount=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedOutputBytes=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_90th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkHitCount=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadOpsInCurThread=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_median=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestRatePerSecond=0.0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_25th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_min=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_num_ops=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_num_ops=2 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_90th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockBytesScannedCount=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_max=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_75th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_max=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_mean=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_num_ops=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_25th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheEvictionCount=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99.9th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheCount=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:smallCompactionQueueLength=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCount=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_min=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_num_ops=6 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_mean=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowIncrementCount=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushQueueLength=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_mean=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_75th_percentile=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:localBytesRead=123986 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushCount=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_median=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutateFailedCount=0 2023-10-15T16:24:17,138 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_max=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorPoolAllocationBytes=199680 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_98th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_min=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_median=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_min=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:updatesBlockedTime=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_median=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_mean=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitQueueLength=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_min=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_min=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_95th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCachingCount=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_25th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99.9th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileAge=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_median=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_75th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_max=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_95th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_num_ops=1 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_min=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cpRequestCount=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_90th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_median=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_max=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_num_ops=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALCount=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_98th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitRatio=0.0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataMissCount=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestCount=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissCount=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_75th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_mean=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheEvictionCount=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheFreeSize=922070024 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_95th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_median=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_75th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_num_ops=1 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_max=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_median=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:activeScanners=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissCount=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_mean=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_max=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_max=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_95th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactionQueueLength=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_95th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_num_ops=1 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_max=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_1min_rate=0.0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_min=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_median=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_num_ops=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_median=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_mean=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_75th_percentile=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_min=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:17,139 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99.9th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_median=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_25th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_25th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_90th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_num_ops=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheSize=676856 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_num_ops=2 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_98th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_count=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_median=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowGetCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_median=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterRequestsCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_min=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_min=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_25th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_max=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileSize=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_min=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99.9th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_98th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_num_ops=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedOutputBytes=5860 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_max=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_num_ops=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_max=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticBloomSize=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexMissCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseInfoThresholdExceeded=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_max=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedToMob=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedMemstoreBytes=928 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheExpressHitPercent=0.0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionServerStartTime=1697385851561 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedInputBytes=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedFromMob=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_max=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRequestCount=11 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_25th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcGetRequestCount=3 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_90th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCountHitPercent=0.0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_max=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_mean=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_min=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_98th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_min=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsSize=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_mean=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowDeleteCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockedRequestCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_mean=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheAccessCount=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_min=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_median=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_min=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_90th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_min=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_95th_percentile=0 2023-10-15T16:24:17,140 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_90th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheEvictedCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_90th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_max=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:avgStoreFileAge=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_25th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitRequestCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_min=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalBytesRead=123986 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedToMob=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_max=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedInputBytes=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheSize=676856 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileCount=479 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_mean=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcScanRequestCount=2 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreOffHeapSize=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_75th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheMissCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowPutCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_max=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreSize=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_75th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsSize=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_mean=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_min=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_75th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_median=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_98th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_median=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Context=regionserver 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:scannerLeaseExpiredCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:minStoreFileAge=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_98th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcFullScanRequestCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_mean=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitSuccessCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_mean=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_75th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_25th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCountPrimary=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_max=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataBlockCount=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,141 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_98th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_max=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_median=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsCount=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_mean=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsCount=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCachingCount=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFailedInsertionCount=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_90th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsSize=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean_rate=0.0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticIndexSize=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_min=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99.9th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissRatio=0.0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_75th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_max=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_mean=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_95th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMultiRequestCount=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheSize=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexHitCount=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_max=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_98th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedOutputBytes=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocal=0.0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_98th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_max=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_num_ops=3 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99.9th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_25th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_min=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_mean=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_15min_rate=0.0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99.9th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_min=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_95th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_75th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_median=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_median=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_mean=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_95th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitCount=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:largeCompactionQueueLength=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_mean=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutatePassedCount=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_max=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_95th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissRatio=0.0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsCount=0 2023-10-15T16:24:17,142 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_min=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_mean=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_min=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:shortCircuitBytesRead=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99.9th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_median=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_min=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoMissCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileIndexSize=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99.9th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_mean=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerHitCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_98th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsSize=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCountPrimary=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_min=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_max=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99.9th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_90th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_90th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_num_ops=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_max=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_mean=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_max=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_min=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRowActionRequestCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMutateRequestCount=6 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_25th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_mean=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataHitCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:zeroCopyBytesRead=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_98th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_num_ops=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_num_ops=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_median=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALSize=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_num_ops=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_mean=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_median=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheFreeSize=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:averageRegionSize=928 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_95th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaMissCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_75th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:filteredReadRequestCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaHitCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_75th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_num_ops=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_95th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_median=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_num_ops=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.serverName=8c5d72df62de,34443,1697385851561 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReads=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowAppendCount=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:numReferenceFiles=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedFromMob=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSize=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_max=0 2023-10-15T16:24:17,143 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_90th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_median=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_mean=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_98th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadWins=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseWarnThresholdExceeded=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_max=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_25th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoHitCount=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_median=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreHeapSize=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_25th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_min=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99.9th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_25th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_num_ops=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_90th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_max=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99.9th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_max=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_median=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsSize=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_median=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_median=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_min=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkMissCount=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_95th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFreeSize=922070024 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheHitPercent=0.0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsCount=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_98th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_median=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestRatePerSecond=0.0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerMissCount=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_mean=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_median=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_98th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_95th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_median=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_90th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_5min_rate=0.0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_min=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_95th_percentile=0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitRatio=0.0 2023-10-15T16:24:17,144 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_75th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_90th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_num_ops=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_min=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_25th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestCount=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedOutputBytes=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_90th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkHitCount=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadOpsInCurThread=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_median=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestRatePerSecond=0.0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_25th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_min=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_num_ops=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_num_ops=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_90th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockBytesScannedCount=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_max=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_75th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_max=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_mean=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_num_ops=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_25th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheEvictionCount=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99.9th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheCount=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:smallCompactionQueueLength=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCount=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_min=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_num_ops=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_mean=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowIncrementCount=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushQueueLength=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_mean=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_75th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:localBytesRead=123986 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushCount=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_median=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutateFailedCount=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_max=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorPoolAllocationBytes=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_98th_percentile=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_min=0 2023-10-15T16:24:17,145 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_median=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_min=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:updatesBlockedTime=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_median=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_mean=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitQueueLength=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_min=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_min=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_95th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCachingCount=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_25th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99.9th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileAge=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_median=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_75th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_max=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_95th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_num_ops=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_min=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cpRequestCount=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_90th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_median=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_max=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_num_ops=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALCount=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_98th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitRatio=0.0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataMissCount=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestCount=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissCount=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_75th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_mean=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheEvictionCount=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheFreeSize=922070024 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_95th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_median=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_75th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_max=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_median=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:activeScanners=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissCount=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_mean=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_max=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_max=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_95th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactionQueueLength=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_95th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_num_ops=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_max=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_1min_rate=0.0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_min=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_median=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_num_ops=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_median=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_mean=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_75th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_min=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99.9th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,146 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_median=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_25th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_25th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_90th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_num_ops=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheSize=676856 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_num_ops=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_98th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_count=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_median=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowGetCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_median=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterRequestsCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_min=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_min=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_25th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_max=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileSize=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_min=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99.9th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_98th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_num_ops=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedOutputBytes=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_max=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_num_ops=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_max=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticBloomSize=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexMissCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseInfoThresholdExceeded=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_max=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedToMob=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedMemstoreBytes=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheExpressHitPercent=0.0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionServerStartTime=1697385851811 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedInputBytes=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedFromMob=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_max=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRequestCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_25th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcGetRequestCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_90th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCountHitPercent=0.0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_max=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_mean=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_min=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_98th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_min=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsSize=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_mean=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowDeleteCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockedRequestCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_mean=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheAccessCount=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_min=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_median=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_min=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_90th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_min=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_95th_percentile=0 2023-10-15T16:24:17,147 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_90th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheEvictedCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_90th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_max=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:avgStoreFileAge=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_25th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitRequestCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_min=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalBytesRead=123986 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedToMob=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_max=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedInputBytes=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheSize=676856 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileCount=240 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_mean=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcScanRequestCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreOffHeapSize=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_75th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheMissCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowPutCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_max=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreSize=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_75th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsSize=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_mean=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_min=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_75th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_median=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_98th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_median=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Context=regionserver 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:scannerLeaseExpiredCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:minStoreFileAge=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_98th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcFullScanRequestCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_mean=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitSuccessCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_mean=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_75th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_25th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCountPrimary=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_max=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataBlockCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_98th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_max=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_median=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_mean=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsCount=0 2023-10-15T16:24:17,148 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCachingCount=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFailedInsertionCount=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_90th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsSize=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean_rate=0.0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticIndexSize=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorUsedBufferCount=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_min=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99.9th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissRatio=0.0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_75th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_max=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_mean=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_95th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMultiRequestCount=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheSize=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexHitCount=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_max=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_98th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedOutputBytes=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocal=0.0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_98th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_max=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_num_ops=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99.9th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_25th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_min=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_mean=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_15min_rate=0.0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99.9th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_min=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_95th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_75th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_median=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_median=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_mean=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_95th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitCount=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:largeCompactionQueueLength=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_mean=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutatePassedCount=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_max=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_95th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissRatio=0.0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsCount=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_min=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_mean=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_min=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:shortCircuitBytesRead=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99.9th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_median=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_min=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoMissCount=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99th_percentile=0 2023-10-15T16:24:17,149 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileIndexSize=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99.9th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_mean=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerHitCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_98th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsSize=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCountPrimary=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_min=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_max=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99.9th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_90th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_90th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_num_ops=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_max=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_mean=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_max=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_min=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRowActionRequestCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMutateRequestCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_25th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_mean=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataHitCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:zeroCopyBytesRead=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_98th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_num_ops=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_num_ops=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_median=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALSize=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_num_ops=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_mean=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_median=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheFreeSize=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:averageRegionSize=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_95th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaMissCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_75th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:filteredReadRequestCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaHitCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_75th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_num_ops=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_95th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_median=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_num_ops=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.serverName=8c5d72df62de,40585,1697385851811 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReads=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowAppendCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:numReferenceFiles=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedFromMob=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSize=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_max=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_90th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_median=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_mean=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_98th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadWins=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseWarnThresholdExceeded=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_max=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_25th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoHitCount=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_median=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreHeapSize=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_25th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_min=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99.9th_percentile=0 2023-10-15T16:24:17,150 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_25th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_num_ops=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_90th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_max=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99.9th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_max=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_median=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsSize=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_median=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_median=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_min=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkMissCount=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_95th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFreeSize=922070024 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheHitPercent=0.0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsCount=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_98th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_median=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestRatePerSecond=0.0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerMissCount=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_mean=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_median=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_98th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_95th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_median=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_90th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_5min_rate=0.0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_min=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_95th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitRatio=0.0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_75th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_90th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_num_ops=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_min=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_25th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestCount=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedOutputBytes=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_90th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkHitCount=1 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadOpsInCurThread=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_median=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestRatePerSecond=0.0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_25th_percentile=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_min=0 2023-10-15T16:24:17,151 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_num_ops=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_num_ops=11 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_90th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockBytesScannedCount=6840 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_max=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_75th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_max=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_mean=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_num_ops=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_25th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheEvictionCount=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99.9th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheCount=4 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:smallCompactionQueueLength=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCount=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_min=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_num_ops=7 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_num_ops=1 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_mean=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowIncrementCount=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushQueueLength=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_mean=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_75th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:localBytesRead=123986 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushCount=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_median=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutateFailedCount=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_max=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorPoolAllocationBytes=599040 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_98th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_min=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_median=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_min=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:updatesBlockedTime=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_median=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_mean=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitQueueLength=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_min=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_min=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_95th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCachingCount=4 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_25th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99.9th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileAge=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_median=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_75th_percentile=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_max=0 2023-10-15T16:24:17,152 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_95th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_num_ops=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_min=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cpRequestCount=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_90th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_median=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_max=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_num_ops=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALCount=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_98th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitRatio=0.72 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataMissCount=6 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestCount=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissCount=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_75th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_mean=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheEvictionCount=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheFreeSize=922061632 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_95th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_median=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_75th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_max=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_median=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:activeScanners=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissCount=7 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_mean=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_max=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_max=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_95th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactionQueueLength=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_95th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_num_ops=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_max=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_1min_rate=0.0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_min=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_median=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_num_ops=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_median=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_mean=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_75th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_min=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99.9th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_median=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_25th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_25th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_90th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_num_ops=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheSize=685248 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_num_ops=7 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_num_ops=11 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_98th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_count=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_median=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowGetCount=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_median=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterRequestsCount=0 2023-10-15T16:24:17,153 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_min=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_min=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_25th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_num_ops=1 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_max=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileSize=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_min=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99.9th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_98th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_num_ops=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedOutputBytes=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_max=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_num_ops=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_max=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticBloomSize=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexMissCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseInfoThresholdExceeded=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCount=4 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCount=7 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_max=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedToMob=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedMemstoreBytes=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheExpressHitPercent=81.81818181818183 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionServerStartTime=1697385888915 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedInputBytes=18105 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedFromMob=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_max=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRequestCount=21 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_25th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcGetRequestCount=1 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_90th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCountHitPercent=72.0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_max=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_mean=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_min=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_98th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_min=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsSize=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_mean=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowDeleteCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockedRequestCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_mean=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheAccessCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_min=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_median=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_min=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_90th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_min=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_95th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_90th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheEvictedCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_90th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_max=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:avgStoreFileAge=12090 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_25th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitRequestCount=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_num_ops=1 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_min=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalBytesRead=123986 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedToMob=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_max=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedInputBytes=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheSize=685248 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,154 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileCount=699 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_mean=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcScanRequestCount=11 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreOffHeapSize=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_75th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheMissCount=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowPutCount=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_max=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreSize=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_75th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsSize=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_mean=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_min=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_75th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_median=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_98th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_median=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Context=regionserver 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:scannerLeaseExpiredCount=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:minStoreFileAge=2959 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitCount=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_98th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcFullScanRequestCount=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_mean=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitSuccessCount=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_mean=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_75th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_25th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCountPrimary=7 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_max=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataBlockCount=3 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_98th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_max=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_median=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsCount=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_mean=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsCount=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCachingCount=18 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFailedInsertionCount=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_90th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsSize=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean_rate=0.0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticIndexSize=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_min=0 2023-10-15T16:24:17,155 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99.9th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissRatio=0.28 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_75th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_max=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_mean=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_95th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMultiRequestCount=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheSize=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexHitCount=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_max=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_98th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedOutputBytes=5921 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocal=0.0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_98th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_max=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_num_ops=1 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99.9th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_25th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_min=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_mean=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_15min_rate=0.0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99.9th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_min=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_95th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_75th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_median=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_median=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_mean=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_95th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitCount=18 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:largeCompactionQueueLength=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_mean=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutatePassedCount=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_max=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_95th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissRatio=0.0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsCount=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_min=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_mean=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_min=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:shortCircuitBytesRead=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99.9th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_median=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_min=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoMissCount=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileIndexSize=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99.9th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_mean=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerHitCount=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_98th_percentile=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsSize=0 2023-10-15T16:24:17,156 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCountPrimary=18 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_min=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_max=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99.9th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_90th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_90th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_num_ops=1 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_max=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_mean=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_max=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_min=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRowActionRequestCount=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMutateRequestCount=8 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheCount=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_25th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationBytes=111031 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_mean=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataHitCount=17 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:zeroCopyBytesRead=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_98th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionCount=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_num_ops=1 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileCount=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_num_ops=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_median=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALSize=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_num_ops=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_mean=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_median=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheFreeSize=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:averageRegionSize=32082 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_95th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaMissCount=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_75th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:filteredReadRequestCount=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaHitCount=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_75th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_num_ops=1 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_95th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_median=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCount=18 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_num_ops=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.serverName=8c5d72df62de,35419,1697385888915 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReads=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowAppendCount=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:numReferenceFiles=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedFromMob=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSize=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_max=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_90th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_median=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_mean=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_98th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadWins=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseWarnThresholdExceeded=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_max=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_25th_percentile=0 2023-10-15T16:24:17,157 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoHitCount=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_median=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreHeapSize=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_25th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_min=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99.9th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_25th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_num_ops=1 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_90th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_max=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99.9th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_max=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_median=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsSize=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_median=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_median=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_min=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkMissCount=1 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_95th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFreeSize=922061632 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheHitPercent=0.0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_98th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_num_ops=2 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.walReaderEditsBufferUsage=80117 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_25th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfHFileRefsQueue=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logReadInBytes=83847 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_75th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfLogQueue=-3 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_90th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99.9th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_num_ops=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedOps=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_min=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedHFiles=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_min=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_mean=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBytes=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_90th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedOps=1000 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedBatches=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_98th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_max=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_95th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedBatches=1 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_mean=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_95th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedHFiles=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_median=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.uncleanlyClosedLogs=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99th_percentile=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.closedLogsWithUnknownFileLength=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_max=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.repeatedLogFileBytes=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedRecoverQueues=5 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Context=regionserver 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedRecoverQueues=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.restartedLogReading=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.ignoredUncleanlyClosedLogContentsInBytes=0 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsRead=40 2023-10-15T16:24:17,158 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.numInitializing=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99.9th_percentile=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_75th_percentile=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsFiltered=38 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_25th_percentile=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedLogs=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.failedBatches=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBatches=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_median=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWrite=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosNumOps=26 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRemoved=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsNumOps=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksCached=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsAvgTime=94.0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalNumOps=9 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesRead=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpAvgTime=6326.928571428572 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesAvgTime=0.0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionTasks=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncCount=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:NumProcessedCommands=10 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpNumOps=27 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromLocalClient=28 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:VolumeFailures=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosNumOps=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesRead=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosAvgTime=561936.0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalReadTime=9 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesRead=52810 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsAvgTime=1.7142857142857142 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromRemoteClient=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksUncached=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DatanodeNetworkErrors=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromLocalClient=27 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksInPendingIBR=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockVerificationFailures=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosNumOps=54 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesWrite=0 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosNumOps=54 2023-10-15T16:24:17,159 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.SessionId=null 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsNumOps=1 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesWritten=620489 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRead=27 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReplicated=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Context=dfs 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksVerified=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromRemoteClient=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosAvgTime=20303.65 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsNumOps=9 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksWritten=28 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsAvgTime=1.0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalWriteTime=12 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcDecodingTimeNanos=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesNumOps=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosNumOps=1285 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpNumOps=28 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosAvgTime=23635.714285714286 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesWritten=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosAvgTime=10588.8 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpNumOps=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumFailedVolumes=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheCapacity=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheUsed=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:FailedStorageLocations=[Ljava.lang.String;@635460a3 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksCached=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:LastVolumeFailureDate=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Capacity=1054589059072 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToUncache=10 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:DfsUsed=1039261 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Remaining=941147415973 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToCache=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numRegionSizeReports=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:24:17,160 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_min=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_num_ops=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_98th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_num_ops=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numNamespaceInQuotaViolation=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_75th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_95th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_90th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_median=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_max=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.namespaceSpaceQuotaOverview=[] 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numTablesInQuotaViolation=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_num_ops=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99.9th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_max=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_98th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_median=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_min=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_max=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Context=master 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99.9th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_95th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_median=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_mean=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_mean=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_max=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_25th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_mean=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_min=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_75th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_mean=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_90th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_min=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_median=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.tableSpaceQuotaOverview=[] 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numSpaceQuotas=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_25th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_num_ops=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenSubmittedCount=14 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignSubmittedCount=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_num_ops=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99.9th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_median=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_max=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_95th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_95th_percentile=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_min=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_mean=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_min=0 2023-10-15T16:24:17,161 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_median=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_min=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveSubmittedCount=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_25th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_mean=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99.9th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_max=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnFS=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerRegions=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_75th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_95th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignSubmittedCount=13 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_75th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_98th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_num_ops=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_90th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignFailedCount=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_num_ops=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_90th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignFailedCount=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_25th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_98th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_98th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveFailedCount=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_mean=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_75th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeFailedCount=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Context=master 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_num_ops=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseSubmittedCount=1 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:holes=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_75th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99.9th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_95th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_95th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeSubmittedCount=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99.9th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_median=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99.9th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_mean=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCount=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenSubmittedCount=1 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseFailedCount=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_max=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:inconsistentRegions=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_90th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenFailedCount=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnRS=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99.9th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_75th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_75th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_min=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_25th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerOpenRegions=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_95th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_25th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:overlaps=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenFailedCount=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_90th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_98th_percentile=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitFailedCount=0 2023-10-15T16:24:17,162 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_max=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_98th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_mean=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_num_ops=13 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_median=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_mean=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCountOverThreshold=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_min=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_75th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_90th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_num_ops=1 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_75th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_95th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_25th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_median=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_25th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitSubmittedCount=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_mean=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritOldestAge=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:emptyRegionInfoRegions=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_max=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_25th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99.9th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_90th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_min=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99.9th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_num_ops=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_max=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_95th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_median=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_max=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_90th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_25th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_25th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_min=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_90th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_95th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_max=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_num_ops=1 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_min=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:operationCount=15 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_98th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:deadServerOpenRegions=3 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_mean=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_median=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_min=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99.9th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_98th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_median=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_max=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_mean=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_num_ops=14 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_98th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_90th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_median=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_98th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_75th_percentile=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumFailedVolumes=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheUsed=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheCapacity=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksCached=0 2023-10-15T16:24:17,163 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:LastVolumeFailureDate=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Capacity=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Context=FSDatasetState 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data2]'} 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:DfsUsed=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Remaining=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToCache=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToUnCache=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalDataFileIos=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalFileIoErrors=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateNumOps=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateNumOps=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateNumOps=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateNumOps=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateNumOps=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateNumOps=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Context=dfs 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalMetadataOperations=0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsNumOps=17 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportNumOps=14 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsNumOps=57 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoNumOps=707 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestNumOps=3 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedNumOps=172 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateNumOps=64 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.port=39027 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeNumOps=3 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportAvgTime=1.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Context=rpcdetailed 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameNumOps=44 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportAvgTime=0.21428571428571427 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteAvgTime=0.25 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateAvgTime=5.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteNumOps=64 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingNumOps=219 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsNumOps=334 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportNumOps=4 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeAvgTime=0.6666666666666666 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionNumOps=1 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockNumOps=53 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteNumOps=41 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateNumOps=1 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesNumOps=16 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsAvgTime=0.0 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeNumOps=1 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatNumOps=1200 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseNumOps=39 2023-10-15T16:24:17,164 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseNumOps=5 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseAvgTime=0.0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatAvgTime=0.0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemMaxM=2200.0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTerminated=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsWaiting=87 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.SessionId=null 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Context=jvm 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapUsedM=193.45564 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTotalExtraSleepTime=55 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Old Generation=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCount=11 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsBlocked=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogWarn=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsRunnable=84 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogInfo=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsNew=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapUsedM=566.47656 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTimedWaiting=118 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapMaxM=2200.0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Young Generation=11 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogError=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.ProcessName=DataNode 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogFatal=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillis=328 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDataNodeMessageCount=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastWrittenTransactionId=466 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:SnapshottableDirectories=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEncryptionZones=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReconstructionBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastCheckpointTime=1697385856136 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ReadLockLongHoldCount=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumActiveClients=1 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastLogRoll=466 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumTimedOutPendingReconstructions=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:VolumeFailuresTotal=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ProvidedCapacityTotal=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingECBlockGroups=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedGB=0.0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecommissioningDataNodes=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.HAState=active 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyECBlockGroups=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:FilesTotal=109 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ScheduledReplicationBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureECBlockGroups=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptECBlockGroups=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MillisSinceLastLoadedEdits=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalECBlockGroups=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExcessBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalLoad=9 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExpiredHeartbeats=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInServiceLiveDataNodes=3 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumStaleStorages=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:WriteLockLongHoldCount=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlocksTotal=43 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:UnderReplicatedBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicatedBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsed=3126472 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalReplicatedBlocks=43 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:StaleDataNodes=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumFilesUnderConstruction=1 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemaining=2823484494063 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PostponedMisreplicatedBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedNonDFS=178715290424 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LockQueueLength=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptReplicatedBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Context=dfs 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomDeadDataNodes=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:Snapshots=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionECBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.TotalSyncTimes=11 5 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReplicationBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,165 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotal=3163767177216 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDeadDataNodes=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplOneBlocks=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalSyncCount=417 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastCheckpoint=466 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicationOneBlocks=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumLiveDataNodes=3 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlockCapacity=4194304 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotalGB=2946.0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyBlocks=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomLiveDataNodes=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemainingGB=2630.0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:ReceivedBytes=15456 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcSlowCalls=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationSuccesses=46 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumDroppedConnections=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcClientBackoff=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationFailures=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumOpenConnections=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.port=35085 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:SentBytes=9822 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeNumOps=46 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationSuccesses=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:CallQueueLength=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Context=rpc 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationFailures=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeNumOps=46 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeNumOps=46 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:receivedBytes=288613 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationSuccesses=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_max=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFailures=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numOpenConnections=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_num_ops=1195 2023-10-15T16:24:17,166 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_median=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationSuccesses=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_num_ops=1195 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_num_ops=1195 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_max=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_max=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_median=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.otherExceptions=5 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Context=regionserver 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_median=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_min=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_median=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_max=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_min=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_min=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveHandler=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_num_ops=1195 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_mean=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReadHandler=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_median=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_num_ops=1195 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_mean=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callDropped=0 2023-10-15T16:24:17,167 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions=5 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFallbacks=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationFailures=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_min=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_max=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:queueSize=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveScanHandler=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_min=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:sentBytes=44933 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_mean=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_75th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_max=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_90th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_min=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_median=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_num_ops=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOnHeapSize=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_num_ops=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_num_ops=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_median=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_num_ops=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_75th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_min=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOffHeapSize=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_25th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_75th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_mean=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_25th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_25th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_mean=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tunerDoNothingCounter=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_25th_percentile=0 2023-10-15T16:24:17,168 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_num_ops=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:aboveHeapOccupancyLowWaterMarkCounter=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_98th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreSize=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_min=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_max=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_90th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_mean=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_75th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Context=regionserver 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99.9th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_mean=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_min=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockCacheSize=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_90th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockedFlushGauge=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_max=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_median=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_98th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:unblockedFlushGauge=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_max=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_max=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_num_ops=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_min=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_95th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_90th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_95th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_max=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_median=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_98th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_min=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99.9th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_median=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_median=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_95th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_mean=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_95th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_98th_percentile=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalDataFileIos=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalFileIoErrors=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateNumOps=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,169 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateNumOps=0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateNumOps=0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateNumOps=0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateNumOps=0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateNumOps=0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Context=dfs 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalMetadataOperations=0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:HttpPort=null 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XceiverCount=3 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"2","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"1199"}] 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:ClusterId=testClusterID 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3":{"numBlocks":21,"usedSpace":259482,"freeSpace":470506468773,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":134216283},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4":{"numBlocks":22,"usedSpace":779779,"freeSpace":470640685056,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0}} 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DatanodeHostname=127.0.0.1 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SecurityEnabled=false 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:RpcPort=0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DataPort=0 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SoftwareVersion=3.2.4 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:17,170 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SlowDisks=null 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XmitsInProgress=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DNStartedTimeInMillis=1697385857021 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsBatchedInSync=50 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsNumOps=416 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsAvgTime=0.0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportNumOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeNumOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteSnapshotOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsAvgTime=0.05714285714285714 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AllowSnapshotOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:RenameSnapshotOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeAvgTime=0.05 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesCreated=176 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesAppended=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportAvgTime=0.0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageAvgTime=0.0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeNumOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeNumOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeNumOps=246 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeAvgTime=0.0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumTimesReReplicationNotScheduled=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SafeModeTime=1 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalNumOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetListingOps=138 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.ProcessName=NameNode 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageNumOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TotalFileOps=1372 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSymlinkOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AddBlockOps=53 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DisallowSnapshotOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteFileOps=31 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsNumOps=466 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSnapshotOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PendingEditsCount=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ListSnapshottableDirOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockReceivedAndDeletedOps=172 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.SessionId=null 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditNumOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FsImageLoadTime=23 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateFileOps=64 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Context=dfs 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesRenamed=44 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TimeoutReReplications=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsQueued=1 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedNumOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageNumOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageAvgTime=0.0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesInGetListingOps=154 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FileInfoOps=707 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetLinkTargetOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetAdditionalDatanodeOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsBatched=71 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesDeleted=47 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SuccessfulReReplications=23 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetBlockLocations=334 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesTruncated=1 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SnapshotDiffReportOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportNumOps=8 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeNumOps=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditAvgTime=0.0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportAvgTime=0.5 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumFailedVolumes=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheUsed=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheCapacity=0 2023-10-15T16:24:17,171 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksCached=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:LastVolumeFailureDate=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Capacity=1054589059072 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Context=FSDatasetState 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:DfsUsed=1039261 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Remaining=941156091301 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToCache=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToUnCache=10 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalDataFileIos=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalFileIoErrors=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateNumOps=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateNumOps=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateNumOps=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateNumOps=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateNumOps=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateNumOps=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Context=dfs 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalMetadataOperations=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessNumOps=1 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureNumOps=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailures=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsNumOps=4 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailuresTotal=0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Context=ugi 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsAvgTime=29.0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessAvgTime=10.0 2023-10-15T16:24:17,172 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureAvgTime=0.0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfSnapshottableDirs=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheCapacity=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheUsed=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ProvidedCapacity=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DistinctVersionCount=1 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocksWithReplicationFactorOne=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentRemaining=89.24402 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentUsed=9.8821176E-5 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:TotalBlocks=43 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NonDfsUsedSpace=178727004984 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Safemode= 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:UpgradeFinalized=true 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:EnteringMaintenanceNodes={} 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentBlockPoolUsed=9.8821176E-5 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:LiveNodes={"127.0.0.1:40557":{"infoAddr":"127.0.0.1:35357","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:40557","lastContact":2,"usedSpace":1039261,"adminState":"In Service","nonDfsUsedSpace":59571348579,"capacity":1054589059072,"numBlocks":42,"version":"3.2.4","used":1039261,"remaining":941161915813,"blockScheduled":1,"blockPoolUsed":1039261,"blockPoolUsedPercent":9.854653E-5,"volfails":0,"lastBlockReport":19},"127.0.0.1:46365":{"infoAddr":"127.0.0.1:35919","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46365","lastContact":0,"usedSpace":1039261,"adminState":"In Service","nonDfsUsedSpace":59583407203,"capacity":1054589059072,"numBlocks":42,"version":"3.2.4","used":1039261,"remaining":941149857189,"blockScheduled":1,"blockPoolUsed":1039261,"blockPoolUsedPercent":9.854653E-5,"volfails":0,"lastBlockReport":8},"127.0.0.1:46399":{"infoAddr":"127.0.0.1:44887","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46399","lastContact":1,"usedSpace":1047950,"adminState":"In Service","nonDfsUsedSpace":59572249202,"capacity":1054589059072,"numBlocks":42,"version":"3.2.4","used":1047950,"remaining":941161006501,"blockScheduled":1,"blockPoolUsed":1047950,"blockPoolUsedPercent":9.937046E-5,"volfails":0,"lastBlockReport":19}} 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CompileInfo=2022-07-12T11:58Z by ubuntu from branch-3.2.4 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolId=BP-1531809096-172.17.0.3-1697385856100 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ClusterId=testClusterID 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DeadNodes={} 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:JournalTransactionInfo={"MostRecentCheckpointTxId":"0","LastAppliedOrWrittenTxId":"466"} 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Free=2823472779503 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolUsedSpace=3126472 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Threads=289 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocks=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirStatuses={"active":{"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":"IMAGE_AND_EDITS","/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":"IMAGE_AND_EDITS"},"failed":{}} 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:SoftwareVersion=3.2.4 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:VerifyECWithTopologyResult={"isSupported":"false","resultMessage":"The number of DataNodes (3) is less than the minimum required number of DataNodes (9) for the erasure coding policies: RS-6-3-1024k"} 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Used=3126472 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Total=3163767177216 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NodeUsage={"nodeUsage":{"min":"0.00%","median":"0.00%","max":"0.00%","stdDev":"0.00%"}} 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NNStartedTimeInMillis=1697385856352 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DecomNodes={} 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CorruptFiles=[] 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirSize={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":737,"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":737} 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameJournalStatus=[{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"},{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"}] 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemMaxM=2200.0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTerminated=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsWaiting=87 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.SessionId=null 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Context=jvm 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapUsedM=193.46402 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTotalExtraSleepTime=55 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Old Generation=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCount=11 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsBlocked=0 2023-10-15T16:24:17,174 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogWarn=0 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsRunnable=84 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogInfo=0 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsNew=0 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapUsedM=567.47656 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTimedWaiting=118 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapMaxM=2200.0 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Young Generation=11 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogError=0 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.ProcessName=DataNode 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogFatal=0 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillis=328 2023-10-15T16:24:17,175 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlocksTotal=43 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TopUserOpCounts={"timestamp":"2023-10-15T16:24:17+0000","windows":[{"ops":[{"opType":"*","totalCount":0,"allUsers":[],"topUsers":[]}],"windowLenMs":60000},{"ops":[{"opType":"*","totalCount":53,"allUsers":[{"user":"jenkins","count":53}],"topUsers":[{"user":"jenkins","count":53}]},{"opType":"getfileinfo","totalCount":31,"allUsers":[{"user":"jenkins","count":31}],"topUsers":[{"user":"jenkins","count":31}]},{"opType":"rename","totalCount":3,"allUsers":[{"user":"jenkins","count":3}],"topUsers":[{"user":"jenkins","count":3}]},{"opType":"mkdirs","totalCount":3,"allUsers":[{"user":"jenkins","count":3}],"topUsers":[{"user":"jenkins","count":3}]},{"opType":"create","totalCount":4,"allUsers":[{"user":"jenkins","count":4}],"topUsers":[{"user":"jenkins","count":4}]},{"opType":"open","totalCount":12,"allUsers":[{"user":"jenkins","count":12}],"topUsers":[{"user":"jenkins","count":12}]}],"windowLenMs":300000},{"ops":[{"opType":"*","totalCount":1499,"allUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":375},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":375},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}]},{"opType":"datanodeReport","totalCount":14,"allUsers":[{"user":"jenkins","count":14}],"topUsers":[{"user":"jenkins","count":14}]},{"opType":"listStatus","totalCount":219,"allUsers":[{"user":"jenkins","count":79},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}],"topUsers":[{"user":"jenkins","count":79},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}]},{"opType":"safemode_get","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"delete","totalCount":41,"allUsers":[{"user":"jenkins","count":20},{"user":"jenkins.hfs.4","count":13},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}],"topUsers":[{"user":"jenkins","count":20},{"user":"jenkins.hfs.4","count":13},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}]},{"opType":"getfileinfo","totalCount":707,"allUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":168},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}],"topUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":168},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}]},{"opType":"truncate","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"rename","totalCount":44,"allUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":15},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":15},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}]},{"opType":"mkdirs","totalCount":57,"allUsers":[{"user":"jenkins","count":35},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}],"topUsers":[{"user":"jenkins","count":35},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}]},{"opType":"create","totalCount":64,"allUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":21},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":21},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}]},{"opType":"setPermission","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"setTimes","totalCount":16,"allUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}]},{"opType":"open","totalCount":334,"allUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":16}],"topUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":16}]}],"windowLenMs":1500000}]} 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:SnapshotStats={"SnapshottableDirectories":0,"Snapshots":0} 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:UnderReplicatedBlocks=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityUsed=3126472 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEncryptionZones=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityRemaining=2823472779503 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReconstructionBlocks=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomDeadDataNodes=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FSState=Operational 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:VolumeFailuresTotal=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ProvidedCapacityTotal=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecommissioningDataNodes=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReplicationBlocks=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncTimes=11 5 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingDeletionBlocks=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityTotal=3163767177216 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FilesTotal=109 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ScheduledReplicationBlocks=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDeadDataNodes=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncCount=417 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FsLockQueueLength=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:MaxObjects=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalLoad=9 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleDataNodes=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumLiveDataNodes=3 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlockDeletionStartTime=1697385856352 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInServiceLiveDataNodes=3 2023-10-15T16:24:17,176 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:LowRedundancyBlocks=0 2023-10-15T16:24:17,177 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomLiveDataNodes=0 2023-10-15T16:24:17,177 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleStorages=0 2023-10-15T16:24:17,177 INFO [pool-261-thread-1 {}] common.MetricsLoggerTask(109): << End DataNode metrics dump 2023-10-15T16:24:17,309 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(83): >> Begin DataNode metrics dump 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_num_ops=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_mean=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_num_ops=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_25th_percentile=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_95th_percentile=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_90th_percentile=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_90th_percentile=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_max=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_98th_percentile=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_25th_percentile=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_75th_percentile=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_min=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99th_percentile=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_75th_percentile=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_min=0 2023-10-15T16:24:17,519 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_max=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_max=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_25th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_99.9th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_75th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_mean=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99.9th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_75th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_median=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_95th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_98th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_99th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99.9th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_median=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_25th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Context=master 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_median=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_95th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_98th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_90th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_mean=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_99.9th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_95th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitSize_min=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_max=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_99th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_98th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_mean=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_num_ops=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:HlogSplitTime_min=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_median=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitSize_num_ops=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=FileSystem:MetaHlogSplitTime_90th_percentile=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:HttpPort=null 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XceiverCount=3 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"0","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"384","LastBlockReport":"486"}] 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:ClusterId=testClusterID 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1":{"numBlocks":21,"usedSpace":259482,"freeSpace":470506313125,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":134216283},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2":{"numBlocks":22,"usedSpace":779779,"freeSpace":470640529408,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0}} 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DatanodeHostname=127.0.0.1 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SecurityEnabled=false 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:RpcPort=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DataPort=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SoftwareVersion=3.2.4 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:SlowDisks=null 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:XmitsInProgress=0 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:DNStartedTimeInMillis=1697385856691 2023-10-15T16:24:17,520 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-3:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Context=regionserver 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Regions:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.port=44717 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Context=rpcdetailed 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:GetReplicaVisibleLengthNumOps=50 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpNumOps=5 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksWrite=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosNumOps=48 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRemoved=10 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalAvgTime=0.8367346938775511 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CacheReportsNumOps=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksCached=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsAvgTime=2.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DataNodeActiveXceiversCount=1 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsTotalNumOps=400 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsNumOps=58 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesRead=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ProcessedCommandsOpAvgTime=50.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpAvgTime=225065.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesAvgTime=0.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionTasks=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncCount=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:NumProcessedCommands=401 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpNumOps=104 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromLocalClient=52 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:VolumeFailures=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosNumOps=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesRead=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:PacketAckRoundTripTimeNanosAvgTime=818128.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalReadTime=18 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesRead=362275 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:IncrementalBlockReportsAvgTime=1.4 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromRemoteClient=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksUncached=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:DatanodeNetworkErrors=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadsFromLocalClient=104 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksInPendingIBR=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockVerificationFailures=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosNumOps=208 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBytesWrite=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosNumOps=208 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.SessionId=null 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockReportsNumOps=1 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BytesWritten=1008141 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksRead=104 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReplicated=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Context=dfs 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksVerified=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WritesFromRemoteClient=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketTransferNanosAvgTime=26277.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsNumOps=400 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksWritten=52 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:HeartbeatsAvgTime=0.7857142857142857 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:TotalWriteTime=21 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcDecodingTimeNanos=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:LifelinesNumOps=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosNumOps=1724 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:WriteBlockOpNumOps=52 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FlushNanosAvgTime=5329.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:RemoteBytesWritten=0 2023-10-15T16:24:17,521 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:SendDataPacketBlockedOnNetworkNanosAvgTime=10111.5 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:ReadBlockOpAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:CopyBlockOpNumOps=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-40557:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumFailedVolumes=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheUsed=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:CacheCapacity=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksCached=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:LastVolumeFailureDate=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Capacity=1054589059072 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Context=FSDatasetState 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:DfsUsed=1039261 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:Remaining=941156091301 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToCache=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-4:NumBlocksFailedToUnCache=10 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheCleared=1 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheHit=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Context=rpc 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:CacheUpdated=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalDataFileIos=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalFileIoErrors=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateNumOps=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateNumOps=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:DataFileIoRateNumOps=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateNumOps=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateNumOps=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:FlushIoRateNumOps=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Context=dfs 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TotalMetadataOperations=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:MissingECBlockGroups=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:LowRedundancyECBlockGroups=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:BytesInFutureECBlockGroups=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:CorruptECBlockGroups=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:PendingDeletionECBlocks=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ECBlockGroupsState-1:TotalECBlockGroups=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheCleared=1 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheHit=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Context=rpc 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:CacheUpdated=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:LastHATransitionTime=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:HostAndPort=localhost:39027 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:State=active 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowPeersReport=null 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SlowDisksReport=null 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:NNRole=NameNode 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:BytesWithFutureGenerationStamps=0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeStatus-1:SecurityEnabled=false 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.port=35753 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Context=rpcdetailed 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:GetReplicaVisibleLengthNumOps=55 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:ReceivedBytes=18480 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:17,522 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcSlowCalls=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationSuccesses=55 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumDroppedConnections=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcClientBackoff=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthorizationFailures=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:NumOpenConnections=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.port=35753 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:SentBytes=11751 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcProcessingTimeNumOps=55 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationSuccesses=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:CallQueueLength=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Context=rpc 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcAuthenticationFailures=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcLockWaitTimeNumOps=55 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:RpcQueueTimeNumOps=55 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35753:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:ReceivedBytes=16800 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcSlowCalls=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationSuccesses=50 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumDroppedConnections=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcClientBackoff=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthorizationFailures=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:NumOpenConnections=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.port=44717 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:SentBytes=10673 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcProcessingTimeNumOps=50 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationSuccesses=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:CallQueueLength=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Context=rpc 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcAuthenticationFailures=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcLockWaitTimeNumOps=50 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:RpcQueueTimeNumOps=50 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort44717:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemMaxM=2200.0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTerminated=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsWaiting=87 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.SessionId=null 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Context=jvm 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapUsedM=193.44197 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTotalExtraSleepTime=51 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Old Generation=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCount=11 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsBlocked=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogWarn=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsRunnable=84 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogInfo=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsNew=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapUsedM=562.47656 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:ThreadsTimedWaiting=118 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:MemHeapMaxM=2200.0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcCountG1 Young Generation=11 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogError=0 2023-10-15T16:24:17,523 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:tag.ProcessName=NameNode 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:LogFatal=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcTimeMillis=328 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumFailedVolumes=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheUsed=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:CacheCapacity=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksCached=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:LastVolumeFailureDate=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Capacity=1054589059072 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Context=FSDatasetState 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:DfsUsed=1047950 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:Remaining=941156091301 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToCache=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5:NumBlocksFailedToUnCache=10 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemMaxM=2200.0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTerminated=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsWaiting=87 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.SessionId=null 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Context=jvm 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapUsedM=193.44412 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTotalExtraSleepTime=22 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Old Generation=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCount=11 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsBlocked=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogWarn=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsRunnable=84 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogInfo=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsNew=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapUsedM=562.47656 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:ThreadsTimedWaiting=118 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:MemHeapMaxM=2200.0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcCountG1 Young Generation=11 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogError=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:tag.ProcessName=DataNode 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:LogFatal=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcTimeMillis=328 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-1:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_num_ops=1906 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99th_percentile=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_max=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_mean=187 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:appendCount=1906 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:lowReplicaRollRequest=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99th_percentile=336 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99.9th_percentile=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:sizeRollRequest=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendSize=301542 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_max=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowAppendCount=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:successfulLogRolls=25 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_99.9th_percentile=337 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_TimeRangeCount_0-1=4 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_25th_percentile=37 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendSize=9557 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_90th_percentile=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendCount=1011 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_25th_percentile=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_10-100=2 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_99th_percentile=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_98th_percentile=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_75th_percentile=297 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_75th_percentile=0 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_max=338 2023-10-15T16:24:17,524 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_95th_percentile=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_mean=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendCount=48 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_SizeRangeCount_100-1000=2 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:errorRollRequest=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:rollRequest=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_num_ops=1819 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_median=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_98th_percentile=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_min=36 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_min=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_median=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Context=regionserver 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_TimeRangeCount_0-1=4 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:replication.appendCount=45 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_95th_percentile=329 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncRep.appendSize=74104 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_median=38 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_98th_percentile=334 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_25th_percentile=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_75th_percentile=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_mean=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_99.9th_percentile=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:hbase:meta.appendSize=14494 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:writtenBytes=399697 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_90th_percentile=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendTime_95th_percentile=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:slowSyncRollRequest=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_num_ops=1906 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:AppendSize_90th_percentile=321 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:SyncTime_min=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=WAL:master:store.appendCount=802 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicationOneBlocks=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:CorruptReplicatedBlocks=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:MissingReplicatedBlocks=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): ReplicatedBlocksState-1:TotalReplicatedBlocks=43 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalDataFileIos=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalFileIoErrors=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateNumOps=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateNumOps=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:DataFileIoRateNumOps=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateNumOps=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateNumOps=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:FlushIoRateNumOps=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Context=dfs 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TotalMetadataOperations=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotNumOps=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishNumOps=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:SnapshotAvgTime=0.0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Context=metricssystem 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSources=67 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:DroppedPubAll=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:PublishAvgTime=0.0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSources=67 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumAllSinks=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): MetricsSystem,sub=Stats:NumActiveSinks=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheCleared=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheHit=0 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Context=rpc 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:CacheUpdated=149 2023-10-15T16:24:17,525 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksWrite=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosNumOps=28 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRemoved=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CacheReportsNumOps=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksCached=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsAvgTime=3.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsTotalNumOps=9 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesRead=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ProcessedCommandsOpAvgTime=9.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpAvgTime=6810.7692307692305 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesAvgTime=0.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionTasks=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncCount=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:NumProcessedCommands=10 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpNumOps=31 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromLocalClient=28 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:VolumeFailures=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosNumOps=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesRead=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:PacketAckRoundTripTimeNanosAvgTime=554484.75 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalReadTime=7 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesRead=57958 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:IncrementalBlockReportsAvgTime=1.5714285714285714 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromRemoteClient=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksUncached=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:DatanodeNetworkErrors=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadsFromLocalClient=31 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksInPendingIBR=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockVerificationFailures=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosNumOps=62 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBytesWrite=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosNumOps=62 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.SessionId=null 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockReportsNumOps=1 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BytesWritten=620489 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksRead=31 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReplicated=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Context=dfs 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksVerified=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WritesFromRemoteClient=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketTransferNanosAvgTime=19658.18181818182 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsNumOps=9 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksWritten=28 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:HeartbeatsAvgTime=1.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:TotalWriteTime=12 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcDecodingTimeNanos=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:LifelinesNumOps=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosNumOps=1285 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:WriteBlockOpNumOps=28 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FlushNanosAvgTime=4301.205882352941 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:RemoteBytesWritten=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:SendDataPacketBlockedOnNetworkNanosAvgTime=9971.454545454546 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:ReadBlockOpAvgTime=0.8181818181818182 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:CopyBlockOpNumOps=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-34495:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemMaxM=2200.0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTerminated=0 2023-10-15T16:24:17,526 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsWaiting=87 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.SessionId=null 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Context=jvm 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapUsedM=193.44412 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTotalExtraSleepTime=62 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Old Generation=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCount=11 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsBlocked=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogWarn=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsRunnable=84 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogInfo=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsNew=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapUsedM=562.47656 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:ThreadsTimedWaiting=118 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:MemHeapMaxM=2200.0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcCountG1 Young Generation=11 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogError=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:tag.ProcessName=DataNode 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:LogFatal=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcTimeMillis=328 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-2:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Context=regionserver 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Tables:numTables=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99.9th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_90th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_95th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterRequests=1101 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_99th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterWriteRequests=1059 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_mean=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:clusterReadRequests=74 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_min=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_25th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashSubmittedCount=4 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashFailedCount=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_98th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Context=master 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_num_ops=2 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_75th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_max=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Server:ServerCrashTime_median=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:receivedBytes=6051329 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_98th_percentile=1 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationSuccesses=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_25th_percentile=16 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_max=1 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_98th_percentile=16 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_90th_percentile=1 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFailures=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_98th_percentile=1 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numOpenConnections=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_75th_percentile=16 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_num_ops=1334 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99.9th_percentile=1 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_median=36 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_90th_percentile=1 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationSuccesses=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99.9th_percentile=1 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_max=1 2023-10-15T16:24:17,527 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_num_ops=1334 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_num_ops=1334 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_max=16 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_max=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_median=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.otherExceptions=3 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_95th_percentile=16 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Context=master 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99th_percentile=16 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_median=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_min=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99.9th_percentile=1 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_median=16 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_max=1 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_95th_percentile=1 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_min=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_98th_percentile=36 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_min=16 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_SizeRangeCount_10-100=30 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_99.9th_percentile=16 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_99th_percentile=1 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveHandler=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_num_ops=1334 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_mean=16 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_SizeRangeCount_10-100=30 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveReadHandler=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_median=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_90th_percentile=36 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_num_ops=1334 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_99th_percentile=1 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_mean=36 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_95th_percentile=1 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_25th_percentile=36 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.callDropped=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_75th_percentile=36 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions=3 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authenticationFallbacks=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_99th_percentile=1 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:authorizationFailures=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_min=36 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_95th_percentile=1 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_TimeRangeCount_0-1=30 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_max=36 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:queueSize=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99.9th_percentile=36 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:numActiveScanHandler=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_95th_percentile=36 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:TotalCallTime_98th_percentile=1 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ResponseSize_90th_percentile=16 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:RequestSize_99th_percentile=36 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:QueueCallTime_min=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:24:17,528 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=IPC:sentBytes=18267 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumFailedVolumes=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheUsed=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:CacheCapacity=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksCached=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:LastVolumeFailureDate=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Capacity=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Context=FSDatasetState 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data4]'} 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:DfsUsed=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:Remaining=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToCache=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState:NumBlocksFailedToUnCache=0 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:HttpPort=null 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XceiverCount=3 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"1","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"1198"}] 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:ClusterId=testClusterID 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5":{"numBlocks":22,"usedSpace":274432,"freeSpace":470506173861,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":134216283},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6":{"numBlocks":21,"usedSpace":773518,"freeSpace":470640390144,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0}} 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DatanodeHostname=127.0.0.1 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SecurityEnabled=false 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:17,529 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:RpcPort=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DataPort=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SoftwareVersion=3.2.4 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:SlowDisks=null 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:XmitsInProgress=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:DNStartedTimeInMillis=1697385857417 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-5:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalDataFileIos=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalFileIoErrors=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateNumOps=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateNumOps=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:DataFileIoRateNumOps=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateNumOps=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateNumOps=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:FlushIoRateNumOps=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Context=dfs 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TotalMetadataOperations=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumFailedVolumes=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheUsed=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:CacheCapacity=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksCached=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:LastVolumeFailureDate=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Capacity=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Context=FSDatasetState 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data6]'} 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:DfsUsed=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:Remaining=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToCache=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-2:NumBlocksFailedToUnCache=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:ReceivedBytes=1072216 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcSlowCalls=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationSuccesses=47 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumDroppedConnections=0 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.NumOpenConnectionsPerUser={"jenkins":1} 2023-10-15T16:24:17,530 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcClientBackoff=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthorizationFailures=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:NumOpenConnections=1 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.port=39027 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:SentBytes=679332 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcProcessingTimeNumOps=3059 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationSuccesses=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:CallQueueLength=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Context=rpc 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcAuthenticationFailures=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcLockWaitTimeNumOps=3059 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.serverName=ClientNamenodeProtocol 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:RpcQueueTimeNumOps=3059 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort39027:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.port=35085 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthAvgTime=0.0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Context=rpcdetailed 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:GetReplicaVisibleLengthNumOps=46 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemMaxM=2200.0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTerminated=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsWaiting=87 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.SessionId=null 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Context=jvm 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapUsedM=193.44489 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTotalExtraSleepTime=45 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Old Generation=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCount=11 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsBlocked=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogWarn=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsRunnable=84 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogInfo=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsNew=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapUsedM=562.47656 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:ThreadsTimedWaiting=118 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:MemHeapMaxM=2200.0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcCountG1 Young Generation=11 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogError=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:tag.ProcessName=DataNode 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:LogFatal=0 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcTimeMillis=328 2023-10-15T16:24:17,531 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-7:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsBatchedInSync=31 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsNumOps=224 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SyncsAvgTime=0.13043478260869565 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportNumOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeNumOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteSnapshotOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsAvgTime=0.03896103896103896 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AllowSnapshotOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:RenameSnapshotOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeAvgTime=0.0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesCreated=80 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesAppended=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CacheReportAvgTime=0.0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageAvgTime=0.0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeNumOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogFetchTimeNumOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ResourceCheckTimeNumOps=3 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailTimeAvgTime=0.0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumTimesReReplicationNotScheduled=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SafeModeTime=3 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:EditLogTailIntervalNumOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetListingOps=66 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.ProcessName=NameNode 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetImageNumOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TotalFileOps=540 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSymlinkOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:AddBlockOps=28 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DisallowSnapshotOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:DeleteFileOps=5 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TransactionsNumOps=255 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateSnapshotOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PendingEditsCount=1 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:ListSnapshottableDirOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockReceivedAndDeletedOps=96 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.SessionId=null 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditNumOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FsImageLoadTime=173 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:CreateFileOps=36 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Context=dfs 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesRenamed=17 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:TimeoutReReplications=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsQueued=1 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:NumEditLogLoadedNumOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageNumOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:PutImageAvgTime=0.0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesInGetListingOps=35 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FileInfoOps=270 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetLinkTargetOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetAdditionalDatanodeOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:BlockOpsBatched=9 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesDeleted=1 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SuccessfulReReplications=2 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetBlockLocations=118 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:FilesTruncated=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:SnapshotDiffReportOps=0 2023-10-15T16:24:17,532 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportNumOps=6 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GenerateEDEKTimeNumOps=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:GetEditAvgTime=0.0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:StorageBlockReportAvgTime=0.5 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumFailedVolumes=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheCapacity=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:CacheUsed=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:FailedStorageLocations=[Ljava.lang.String;@7242030b 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksCached=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:LastVolumeFailureDate=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Capacity=1054589059072 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToUncache=10 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:DfsUsed=1039261 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:Remaining=941146301861 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-5acfdfa5-d523-41fd-87ab-72d83e598754:NumBlocksFailedToCache=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpNumOps=6 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksWrite=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosNumOps=54 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRemoved=10 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalAvgTime=0.7474747474747475 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CacheReportsNumOps=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksCached=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsAvgTime=2.0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DataNodeActiveXceiversCount=1 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsTotalNumOps=400 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsNumOps=55 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesRead=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ProcessedCommandsOpAvgTime=34.0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpAvgTime=225066.0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesAvgTime=0.0 2023-10-15T16:24:17,533 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionTasks=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncCount=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:NumProcessedCommands=401 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpNumOps=99 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromLocalClient=52 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:VolumeFailures=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosNumOps=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesRead=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:PacketAckRoundTripTimeNanosAvgTime=924152.0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalReadTime=29 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesRead=688752 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:IncrementalBlockReportsAvgTime=2.0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromRemoteClient=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksUncached=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:DatanodeNetworkErrors=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadsFromLocalClient=99 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksInPendingIBR=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockVerificationFailures=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosNumOps=200 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBytesWrite=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosNumOps=200 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.SessionId=null 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockReportsNumOps=1 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BytesWritten=1008141 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksRead=99 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReplicated=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Context=dfs 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksVerified=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WritesFromRemoteClient=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketTransferNanosAvgTime=45642.0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsNumOps=400 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksWritten=52 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:HeartbeatsAvgTime=0.696969696969697 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:TotalWriteTime=22 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcDecodingTimeNanos=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:LifelinesNumOps=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosNumOps=1724 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:WriteBlockOpNumOps=52 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FlushNanosAvgTime=6378.0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:RemoteBytesWritten=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:SendDataPacketBlockedOnNetworkNanosAvgTime=9294.0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:CopyBlockOpNumOps=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,534 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46399:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksWrite=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosNumOps=30 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRemoved=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CacheReportsNumOps=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksCached=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsAvgTime=95.0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsTotalNumOps=9 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesRead=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpAvgTime=6809.0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesAvgTime=0.0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionTasks=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncCount=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:NumProcessedCommands=10 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpNumOps=35 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromLocalClient=28 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:VolumeFailures=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosNumOps=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesRead=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:PacketAckRoundTripTimeNanosAvgTime=352052.375 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalReadTime=16 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesRead=75811 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:IncrementalBlockReportsAvgTime=1.3846153846153846 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromRemoteClient=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksUncached=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:DatanodeNetworkErrors=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadsFromLocalClient=35 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksInPendingIBR=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockVerificationFailures=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosNumOps=70 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBytesWrite=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosNumOps=70 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.SessionId=null 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockReportsNumOps=1 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,535 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BytesWritten=620489 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksRead=35 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReplicated=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Context=dfs 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksVerified=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WritesFromRemoteClient=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketTransferNanosAvgTime=25843.916666666668 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsNumOps=9 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksWritten=28 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:HeartbeatsAvgTime=1.0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:TotalWriteTime=12 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcDecodingTimeNanos=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:LifelinesNumOps=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosNumOps=1285 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:WriteBlockOpNumOps=28 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FlushNanosAvgTime=4740.882352941177 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:RemoteBytesWritten=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:SendDataPacketBlockedOnNetworkNanosAvgTime=9895.041666666666 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:ReadBlockOpAvgTime=0.75 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:CopyBlockOpNumOps=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-33713:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpNumOps=6 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksWrite=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosNumOps=34 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRemoved=10 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalAvgTime=0.8877551020408163 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CacheReportsNumOps=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksCached=0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsAvgTime=3.0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DataNodeActiveXceiversCount=1 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsTotalNumOps=400 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsNumOps=59 2023-10-15T16:24:17,536 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesRead=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ProcessedCommandsOpAvgTime=10.0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpAvgTime=225063.25 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesAvgTime=0.0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionTasks=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncCount=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:NumProcessedCommands=402 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpNumOps=91 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromLocalClient=52 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:VolumeFailures=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosNumOps=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesRead=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:PacketAckRoundTripTimeNanosAvgTime=1214340.0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalReadTime=12 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesRead=349472 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:IncrementalBlockReportsAvgTime=1.0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromRemoteClient=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksUncached=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:DatanodeNetworkErrors=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadsFromLocalClient=91 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksInPendingIBR=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockVerificationFailures=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosNumOps=182 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBytesWrite=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosNumOps=182 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.SessionId=null 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockReportsNumOps=2 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BytesWritten=1008141 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksRead=91 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReplicated=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Context=dfs 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksVerified=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WritesFromRemoteClient=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketTransferNanosAvgTime=53154.75 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsNumOps=400 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksWritten=52 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:HeartbeatsAvgTime=0.8571428571428571 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:TotalWriteTime=28 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcDecodingTimeNanos=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:LifelinesNumOps=0 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosNumOps=1724 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:WriteBlockOpNumOps=52 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FlushNanosAvgTime=5041.5 2023-10-15T16:24:17,537 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:RemoteBytesWritten=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:SendDataPacketBlockedOnNetworkNanosAvgTime=8707.0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:CopyBlockOpNumOps=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-46365:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDataNodeMessageCount=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastWrittenTransactionId=255 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:SnapshottableDirectories=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEncryptionZones=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReconstructionBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LastCheckpointTime=1697385845476 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ReadLockLongHoldCount=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumActiveClients=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastLogRoll=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumTimedOutPendingReconstructions=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:VolumeFailuresTotal=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ProvidedCapacityTotal=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingECBlockGroups=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedGB=0.0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecommissioningDataNodes=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.HAState=active 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyECBlockGroups=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:FilesTotal=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ScheduledReplicationBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureECBlockGroups=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptECBlockGroups=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MillisSinceLastLoadedEdits=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalECBlockGroups=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExcessBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalLoad=32 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:ExpiredHeartbeats=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumInServiceLiveDataNodes=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumStaleStorages=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:WriteLockLongHoldCount=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlocksTotal=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:UnderReplicatedBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicatedBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsed=495713 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalReplicatedBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:StaleDataNodes=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumFilesUnderConstruction=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemaining=2816043276636 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PostponedMisreplicatedBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityUsedNonDFS=185085906847 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LockQueueLength=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CorruptReplicatedBlocks=0 2023-10-15T16:24:17,538 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingBlocks=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Context=dfs 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomDeadDataNodes=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:Snapshots=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingDeletionECBlocks=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.TotalSyncTimes= 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:PendingReplicationBlocks=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotal=3163767177216 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDeadDataNodes=3 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplOneBlocks=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TotalSyncCount=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:TransactionsSinceLastCheckpoint=255 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:MissingReplicationOneBlocks=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumLiveDataNodes=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:BlockCapacity=4194304 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityTotalGB=2946.0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:LowRedundancyBlocks=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:NumDecomLiveDataNodes=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem:CapacityRemainingGB=2623.0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Context=regionserver 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheCleared=1 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheHit=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Context=rpc 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:CacheUpdated=65 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RetryCache.NameNodeRetryCache-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemMaxM=2200.0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTerminated=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsWaiting=87 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.SessionId= 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Context=jvm 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapUsedM=193.45454 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Old Generation=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCount=11 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsBlocked=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogWarn=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsRunnable=84 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogInfo=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsNew=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapUsedM=563.47656 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:ThreadsTimedWaiting=118 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:MemHeapMaxM=2200.0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcCountG1 Young Generation=11 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogError=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:tag.ProcessName=Master 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:LogFatal=0 2023-10-15T16:24:17,539 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-4:GcTimeMillis=328 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsCount=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeTotal=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointPercentComplete=1.0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageTotal=1 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsTotal=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointCount=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:ElapsedTime=681 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsPercentComplete=1.0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingEditsElapsedTime=7 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointElapsedTime=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModePercentComplete=1.0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:PercentComplete=1.0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SavingCheckpointTotal=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeCount=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:SafeModeElapsedTime=13 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageElapsedTime=62 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImageCount=1 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress:LoadingFsImagePercentComplete=1.0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumFailedVolumes=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheCapacity=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:CacheUsed=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:FailedStorageLocations=[Ljava.lang.String;@887eaec 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksCached=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:LastVolumeFailureDate=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Capacity=1054589059072 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6]'} 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToUncache=10 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:DfsUsed=1047950 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:Remaining=941146039717 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-255d4b5b-842d-4682-b431-747c9bc0eb04:NumBlocksFailedToCache=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsCount=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeTotal=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointPercentComplete=1.0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageTotal=1 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsTotal=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointCount=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:ElapsedTime=681 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsPercentComplete=1.0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingEditsElapsedTime=7 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointElapsedTime=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModePercentComplete=1.0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:PercentComplete=1.0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SavingCheckpointTotal=0 2023-10-15T16:24:17,540 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeCount=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:SafeModeElapsedTime=13 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageElapsedTime=62 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImageCount=1 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): StartupProgress-1:LoadingFsImagePercentComplete=1.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RackLocalityCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionCountSkewCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaHostCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ServerLocalityCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MemStoreSizeCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_95th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_Overall=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_ReadRequestCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_25th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_98th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_min=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Context=master 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_WriteRequestCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_RegionReplicaRackCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_PrimaryRegionCountSkewCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:miscInvocationCount=18 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_median=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_num_ops=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_75th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_MoveCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_max=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_mean=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_CPRequestCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_TableSkewCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:hbase:ensemble_StoreFileCostFunction=0.0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_90th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:tag.isBalancerActive=true 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Balancer:BalancerCluster_99.9th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_95th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99.9th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_median=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_mean=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_25th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_max=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_95th_percentile=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:17,541 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_num_ops=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_25th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_98th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_min=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_mean=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_90th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_75th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_90th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_mean=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_num_ops=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_95th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_min=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_median=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_num_ops=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_mean=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_98th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_75th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_median=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99.9th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Context=regionserver 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_max=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_num_ops=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_num_ops=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_min=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_90th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_mean=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_95th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_max=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_75th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_75th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_min=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_num_ops=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_99th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_75th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_max=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_min=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_98th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_90th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_max=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_98th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_90th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_75th_percentile=0 2023-10-15T16:24:17,542 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_95th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_25th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_max=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_max=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_98th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_num_ops=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_99.9th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_min=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_98th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_95th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_min=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_median=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:BlockBytesScannedCount=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99.9th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_25th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_median=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_90th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_95th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_num_ops=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_max=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_mean=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_99th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_max=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_min=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_99th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_mean=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_95th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_99.9th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_25th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_99.9th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_90th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_mean=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_95th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_min=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_75th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_max=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_max=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_90th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_max=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_98th_percentile=0 2023-10-15T16:24:17,543 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_25th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_median=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_median=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_75th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_mean=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_99.9th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_median=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_90th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_mean=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_mean=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_75th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_num_ops=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_99.9th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_median=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_75th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_median=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_median=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_98th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_95th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_75th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_95th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_num_ops=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_max=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_98th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_25th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_95th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_99.9th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_num_ops=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_25th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99.9th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_median=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_90th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_25th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_median=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_min=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteTime_max=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_90th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_max=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_75th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_25th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndDeleteTime_98th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_min=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_mean=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_99.9th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanSize_25th_percentile=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_min=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_mean=0 2023-10-15T16:24:17,544 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndPutTime_min=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_98th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutBatchTime_min=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_98th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:AppendTime_99.9th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanTime_median=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:DeleteBatchTime_num_ops=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_median=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:GetTime_90th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:PutTime_min=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:CheckAndMutateTime_25th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=TableRequests_Namespace_master_table_store:IncrementTime_99th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsCount=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_98th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_median=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestRatePerSecond=0.0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerMissCount=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_mean=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_median=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_98th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_95th_percentile=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_median=0 2023-10-15T16:24:17,545 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_90th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_5min_rate=0.0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_min=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_95th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitRatio=0.0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_75th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_90th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_num_ops=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_min=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_25th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:writeRequestCount=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedOutputBytes=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_90th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkHitCount=1 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadOpsInCurThread=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_median=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestRatePerSecond=0.0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_25th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_min=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_num_ops=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_num_ops=26 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_90th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockBytesScannedCount=4389 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_max=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_75th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_max=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_mean=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_num_ops=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_25th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheEvictionCount=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99.9th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheCount=3 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:smallCompactionQueueLength=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheEvictionCount=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_min=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_num_ops=15 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_mean=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowIncrementCount=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushQueueLength=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_mean=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_75th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:localBytesRead=123986 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushCount=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_median=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutateFailedCount=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_max=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorPoolAllocationBytes=1530880 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_98th_percentile=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_min=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_median=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_min=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:17,546 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:updatesBlockedTime=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_median=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_mean=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitQueueLength=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_min=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_min=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_95th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCachingCount=3 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_25th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99.9th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileAge=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_median=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_75th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_max=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_95th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_num_ops=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_min=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cpRequestCount=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_90th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_99th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_median=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_max=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_num_ops=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALCount=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_98th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitRatio=0.7272727272727273 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataMissCount=1 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:readRequestCount=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissCount=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_75th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_mean=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheEvictionCount=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheFreeSize=922067336 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_95th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_median=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_75th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_max=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_median=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:activeScanners=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissCount=3 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_mean=0 2023-10-15T16:24:17,547 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_max=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_max=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_95th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactionQueueLength=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_95th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_num_ops=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_max=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_1min_rate=0.0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_min=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_median=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_num_ops=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_median=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_mean=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_75th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_min=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99.9th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_median=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_25th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_25th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_90th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_num_ops=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheSize=679544 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_num_ops=6 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_num_ops=26 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_98th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_count=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_median=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowGetCount=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_median=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterRequestsCount=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_min=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_min=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_25th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_max=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileSize=836094 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_min=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:maxStoreFileCount=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsCount=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_99.9th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_98th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_num_ops=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedOutputBytes=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_max=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_num_ops=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_max=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticBloomSize=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexMissCount=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseInfoThresholdExceeded=0 2023-10-15T16:24:17,548 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCount=3 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCount=3 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_max=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedToMob=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedMemstoreBytes=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheExpressHitPercent=72.72727272727273 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionServerStartTime=1697385859285 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedInputBytes=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsSizeCompactedFromMob=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_max=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRequestCount=47 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_25th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcGetRequestCount=3 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_90th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeCount=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheCountHitPercent=72.72727272727273 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_max=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_mean=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_min=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_98th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_min=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedCellsSize=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_mean=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowDeleteCount=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockedRequestCount=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_mean=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheAccessCount=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_min=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_median=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_min=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_90th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_99th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_min=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_95th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_99th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_90th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheEvictedCount=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_90th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_max=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:avgStoreFileAge=1819 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_25th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheCount=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitRequestCount=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_num_ops=1 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_min=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalBytesRead=123986 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedToMob=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_max=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedInputBytes=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheSize=679544 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_99th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hlogFileCount=953 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_mean=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcScanRequestCount=26 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreOffHeapSize=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_75th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheMissCount=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:17,549 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowPutCount=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_max=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreSize=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_75th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsSize=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_mean=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_min=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_75th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_median=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_98th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_median=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Context=regionserver 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:scannerLeaseExpiredCount=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:minStoreFileAge=1782 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheHitCount=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_98th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcFullScanRequestCount=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_mean=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:splitSuccessCount=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_mean=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_75th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_25th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMissCountPrimary=3 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_max=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataBlockCount=1 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_98th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_max=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_median=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsCount=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_mean=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsCount=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCachingCount=8 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFailedInsertionCount=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_90th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFlushedCellsSize=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_mean_rate=0.0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:staticIndexSize=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorUsedBufferCount=2 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_min=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_99.9th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheMissRatio=0.2727272727272727 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_75th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_max=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_mean=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_95th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMultiRequestCount=1 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheSize=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheRootIndexHitCount=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_max=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_98th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:compactedOutputBytes=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocal=0.0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_98th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndPut_max=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_num_ops=3 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_99.9th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_25th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_min=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_mean=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_15min_rate=0.0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_99.9th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_min=0 2023-10-15T16:24:17,550 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_95th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_75th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_median=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_median=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_mean=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_95th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l1CacheHitCount=8 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:largeCompactionQueueLength=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_mean=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:checkMutatePassedCount=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_max=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_95th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheMissRatio=0.0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:majorCompactedCellsCount=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushOutputSize_min=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_mean=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_min=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:shortCircuitBytesRead=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_99.9th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:DeleteBatch_99th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_median=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_min=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoMissCount=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_99th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileIndexSize=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_99.9th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_mean=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheTrailerHitCount=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_98th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:flushedCellsSize=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCountPrimary=8 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Put_min=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_max=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_99.9th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_90th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_90th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_num_ops=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_max=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_mean=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_max=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_min=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:totalRowActionRequestCount=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:rpcMutateRequestCount=16 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheCount=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_25th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_mean=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:17,551 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheDataHitCount=7 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:zeroCopyBytesRead=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_98th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:regionCount=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_num_ops=1 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileCount=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_num_ops=1 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_median=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mutationsWithoutWALSize=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_num_ops=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_mean=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_median=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:l2CacheFreeSize=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:averageRegionSize=17474 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Get_95th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaMissCount=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_75th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:filteredReadRequestCount=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheMetaHitCount=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_75th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionInputSize_num_ops=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_95th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputSize_median=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheHitCount=8 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_num_ops=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:tag.serverName=8c5d72df62de,45999,1697385859285 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReads=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:slowAppendCount=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:numReferenceFiles=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:cellsCountCompactedFromMob=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:storeFileSize=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Replay_max=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorHeapAllocationRatio=1.0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionTime_90th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_median=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:SplitTime_mean=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_98th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:hedgedReadWins=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:pauseWarnThresholdExceeded=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:AppendBlockBytesScanned_max=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Delete_25th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFileInfoHitCount=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionTime_median=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:memStoreHeapSize=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Bulkload_25th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_min=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_99.9th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanSize_25th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_num_ops=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Append_90th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_max=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:FlushTime_99.9th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_max=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CompactionOutputSize_median=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobScanCellsSize=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_median=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanTime_median=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PutBatch_min=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheBloomChunkMissCount=2 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:Increment_95th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:blockCacheFreeSize=922067336 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:mobFileCacheHitPercent=0.0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemMaxM=2200.0 2023-10-15T16:24:17,552 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTerminated=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsWaiting=87 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.SessionId=null 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Context=jvm 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapUsedM=193.45564 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTotalExtraSleepTime=46 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Old Generation=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCount=11 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsBlocked=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogWarn=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsRunnable=84 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogInfo=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsNew=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapUsedM=564.47656 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:ThreadsTimedWaiting=118 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:MemHeapMaxM=2200.0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcCountG1 Young Generation=11 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogError=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:tag.ProcessName=NameNode 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:LogFatal=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcTimeMillis=328 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-5:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemMaxM=2200.0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTerminated=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsWaiting=87 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.SessionId=null 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Context=jvm 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapUsedM=193.45564 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTotalExtraSleepTime=54 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Old Generation=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCount=11 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsBlocked=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogWarn=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsRunnable=85 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogInfo=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsNew=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapUsedM=564.47656 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:ThreadsTimedWaiting=117 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:MemHeapMaxM=2200.0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcCountG1 Young Generation=11 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogError=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:tag.ProcessName=DataNode 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:LogFatal=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcTimeMillis=328 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-6:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_90th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_25th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_num_ops=16 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99.9th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_90th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_95th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_median=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_max=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_95th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_98th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_98th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_25th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_TimeRangeCount_1-3=1 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:fsChecksumFailureCount=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_median=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_75th_percentile=2 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_min=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_99th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Context=regionserver 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_max=2 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_min=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_90th_percentile=2 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_75th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_max=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_99.9th_percentile=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_98th_percentile=2 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_mean=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_95th_percentile=2 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_num_ops=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_min=2 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_median=2 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99.9th_percentile=2 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_mean=0 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsWriteTime_num_ops=214 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_25th_percentile=2 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_mean=2 2023-10-15T16:24:17,553 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsReadTime_75th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IO:FsPReadTime_99th_percentile=2 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context=dfs 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.1=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.2=dfs 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.2=8c5d72df62de 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Context.1=dfs 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname.1=8c5d72df62de 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:op=*.TotalCount.2=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NNTopUserOpCounts:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Context=master 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:numMasterWALs=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Procedure:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsCount=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_98th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_median=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestRatePerSecond=0.0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerMissCount=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_mean=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_median=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_98th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_95th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_median=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_90th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_5min_rate=0.0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_min=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_95th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitRatio=0.0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_75th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_90th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_num_ops=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_min=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_25th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:writeRequestCount=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedOutputBytes=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_90th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkHitCount=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadOpsInCurThread=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_median=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestRatePerSecond=0.0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_25th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_min=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_num_ops=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_num_ops=2 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_90th_percentile=0 2023-10-15T16:24:17,554 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockBytesScannedCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_max=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_75th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_max=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_mean=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_num_ops=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_25th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheEvictionCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99.9th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:smallCompactionQueueLength=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheEvictionCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_min=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_num_ops=6 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_mean=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowIncrementCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushQueueLength=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_mean=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_75th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:localBytesRead=123986 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_median=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutateFailedCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_max=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorPoolAllocationBytes=199680 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_98th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_min=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_median=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_min=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:updatesBlockedTime=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_median=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_mean=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitQueueLength=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_min=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_min=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_95th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCachingCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_25th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99.9th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileAge=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_median=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_75th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_max=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_95th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_num_ops=1 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_min=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cpRequestCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_90th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_99th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_median=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_max=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_num_ops=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_98th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitRatio=0.0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataMissCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:readRequestCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_75th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_mean=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheEvictionCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheFreeSize=922070024 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_95th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_median=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_75th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_num_ops=1 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_max=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_median=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:activeScanners=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissCount=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_mean=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:17,555 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_max=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_max=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_95th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactionQueueLength=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_95th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_num_ops=1 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_max=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_1min_rate=0.0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_min=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_median=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_num_ops=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_median=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_mean=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_75th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_min=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99.9th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_median=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_25th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_25th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_90th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_num_ops=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheSize=676856 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_num_ops=2 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_98th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_count=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_median=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowGetCount=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_median=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterRequestsCount=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_min=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_min=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_25th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_max=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileSize=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_min=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:maxStoreFileCount=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsCount=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_99.9th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_98th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_num_ops=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedOutputBytes=5860 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_max=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_num_ops=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_max=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticBloomSize=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexMissCount=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseInfoThresholdExceeded=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCount=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCount=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_max=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedToMob=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedMemstoreBytes=928 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheExpressHitPercent=0.0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionServerStartTime=1697385851561 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedInputBytes=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsSizeCompactedFromMob=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_max=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRequestCount=11 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_25th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcGetRequestCount=3 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_90th_percentile=0 2023-10-15T16:24:17,556 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheCountHitPercent=0.0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_max=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_mean=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_min=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_98th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_min=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedCellsSize=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_mean=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowDeleteCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockedRequestCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_mean=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheAccessCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_min=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_median=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_min=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_90th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_99th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_min=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_95th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_99th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_90th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheEvictedCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_90th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_max=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:avgStoreFileAge=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_25th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitRequestCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_min=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalBytesRead=123986 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedToMob=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_max=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedInputBytes=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheSize=676856 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_99th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hlogFileCount=479 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_mean=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcScanRequestCount=2 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreOffHeapSize=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_75th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheMissCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowPutCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_max=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreSize=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_75th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsSize=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_mean=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_min=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_75th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_median=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_98th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_median=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Context=regionserver 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:scannerLeaseExpiredCount=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:minStoreFileAge=0 2023-10-15T16:24:17,557 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheHitCount=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_98th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcFullScanRequestCount=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_mean=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:splitSuccessCount=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_mean=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_75th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_25th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMissCountPrimary=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_max=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataBlockCount=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_98th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_max=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_median=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsCount=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_mean=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsCount=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCachingCount=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFailedInsertionCount=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_90th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFlushedCellsSize=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_mean_rate=0.0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:staticIndexSize=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_min=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_99.9th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheMissRatio=0.0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_75th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_max=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_mean=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_95th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMultiRequestCount=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheSize=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheRootIndexHitCount=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_max=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_98th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:compactedOutputBytes=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocal=0.0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_98th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndPut_max=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_num_ops=3 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_99.9th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_25th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_min=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_mean=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_15min_rate=0.0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_99.9th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_min=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_95th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_75th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_median=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_median=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_mean=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_95th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l1CacheHitCount=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:largeCompactionQueueLength=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_mean=0 2023-10-15T16:24:17,558 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:checkMutatePassedCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_max=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_95th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheMissRatio=0.0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:majorCompactedCellsCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushOutputSize_min=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_mean=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_min=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:shortCircuitBytesRead=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_99.9th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:DeleteBatch_99th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_median=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_min=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoMissCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_99th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileIndexSize=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_99.9th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_mean=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheTrailerHitCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_98th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:flushedCellsSize=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCountPrimary=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Put_min=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_max=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_99.9th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_90th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_90th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_num_ops=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_max=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_mean=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_max=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_min=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:totalRowActionRequestCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:rpcMutateRequestCount=6 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_25th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_mean=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheDataHitCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:zeroCopyBytesRead=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_98th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:regionCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_num_ops=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_num_ops=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_median=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mutationsWithoutWALSize=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_num_ops=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_mean=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_median=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:l2CacheFreeSize=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:averageRegionSize=928 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Get_95th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaMissCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_75th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:filteredReadRequestCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheMetaHitCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_75th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionInputSize_num_ops=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_95th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputSize_median=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheHitCount=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_num_ops=0 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:tag.serverName=8c5d72df62de,34443,1697385851561 2023-10-15T16:24:17,559 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReads=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:slowAppendCount=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:numReferenceFiles=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:cellsCountCompactedFromMob=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:storeFileSize=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Replay_max=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionTime_90th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_median=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:SplitTime_mean=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_98th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:hedgedReadWins=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:pauseWarnThresholdExceeded=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:AppendBlockBytesScanned_max=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Delete_25th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFileInfoHitCount=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionTime_median=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:memStoreHeapSize=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Bulkload_25th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_min=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_99.9th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanSize_25th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_num_ops=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Append_90th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_max=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:FlushTime_99.9th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_max=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CompactionOutputSize_median=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobScanCellsSize=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_median=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanTime_median=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PutBatch_min=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheBloomChunkMissCount=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:Increment_95th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:blockCacheFreeSize=922070024 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:mobFileCacheHitPercent=0.0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-2:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsCount=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_98th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_median=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestRatePerSecond=0.0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerMissCount=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_mean=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_median=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_98th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_95th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_median=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_90th_percentile=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_5min_rate=0.0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_min=0 2023-10-15T16:24:17,560 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_95th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitRatio=0.0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_75th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_90th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_num_ops=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_min=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_25th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:writeRequestCount=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedOutputBytes=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_90th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkHitCount=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadOpsInCurThread=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_median=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestRatePerSecond=0.0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_25th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_min=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_num_ops=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_num_ops=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_90th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockBytesScannedCount=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_max=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_75th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_max=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_mean=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_num_ops=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_25th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheEvictionCount=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99.9th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheCount=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:smallCompactionQueueLength=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheEvictionCount=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_min=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_num_ops=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_mean=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowIncrementCount=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushQueueLength=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_mean=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_75th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:localBytesRead=123986 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushCount=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_median=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutateFailedCount=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_max=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorPoolAllocationBytes=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_98th_percentile=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_min=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_median=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_min=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:updatesBlockedTime=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_median=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_mean=0 2023-10-15T16:24:17,561 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitQueueLength=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_min=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_min=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_95th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCachingCount=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_25th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99.9th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileAge=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_median=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_75th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_max=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_95th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_num_ops=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_min=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cpRequestCount=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_90th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_99th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_median=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_max=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_num_ops=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALCount=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_98th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitRatio=0.0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataMissCount=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:readRequestCount=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissCount=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_75th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_mean=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheEvictionCount=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheFreeSize=922070024 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_95th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_median=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_75th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_max=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_median=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:activeScanners=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissCount=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_mean=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_max=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_max=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_95th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactionQueueLength=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_95th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_num_ops=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_max=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_1min_rate=0.0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_min=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_median=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_num_ops=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_median=0 2023-10-15T16:24:17,562 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_mean=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_75th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_min=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99.9th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_median=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_25th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_25th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_90th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_num_ops=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheSize=676856 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_num_ops=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_98th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_count=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_median=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowGetCount=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_median=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterRequestsCount=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_min=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_min=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_25th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_max=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileSize=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_min=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:maxStoreFileCount=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsCount=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_99.9th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_98th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_num_ops=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedOutputBytes=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_max=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_num_ops=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_max=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticBloomSize=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexMissCount=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseInfoThresholdExceeded=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,563 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_max=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedToMob=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedMemstoreBytes=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheExpressHitPercent=0.0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionServerStartTime=1697385851811 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedInputBytes=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsSizeCompactedFromMob=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_max=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRequestCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_25th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcGetRequestCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_90th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheCountHitPercent=0.0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_max=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_mean=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_min=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_98th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_min=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedCellsSize=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_mean=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowDeleteCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockedRequestCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_mean=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheAccessCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_min=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_median=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_min=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_90th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_99th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_min=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_95th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_99th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_90th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheEvictedCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_90th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_max=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:avgStoreFileAge=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_25th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitRequestCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_min=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalBytesRead=123986 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedToMob=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_max=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedInputBytes=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheSize=676856 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_99th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hlogFileCount=240 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_mean=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcScanRequestCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreOffHeapSize=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_75th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheMissCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowPutCount=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_max=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreSize=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_75th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsSize=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_mean=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_min=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_75th_percentile=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_median=0 2023-10-15T16:24:17,564 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_98th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_median=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Context=regionserver 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:scannerLeaseExpiredCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:minStoreFileAge=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheHitCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_98th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcFullScanRequestCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_mean=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:splitSuccessCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_mean=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_75th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_25th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMissCountPrimary=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_max=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataBlockCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_98th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_max=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_median=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_mean=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCachingCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFailedInsertionCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_90th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFlushedCellsSize=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_mean_rate=0.0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:staticIndexSize=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorUsedBufferCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_min=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_99.9th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheMissRatio=0.0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_75th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_max=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_mean=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_95th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMultiRequestCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheSize=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheRootIndexHitCount=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_max=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_98th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:compactedOutputBytes=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocal=0.0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_98th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndPut_max=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_num_ops=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_99.9th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_25th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_min=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_mean=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_15min_rate=0.0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:17,565 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_99.9th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_min=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_95th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_75th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_median=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_median=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_mean=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_95th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l1CacheHitCount=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:largeCompactionQueueLength=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_mean=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:checkMutatePassedCount=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_max=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_95th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.clusterId=241aca37-6ead-421a-9a95-62a3de5ca646 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheMissRatio=0.0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:majorCompactedCellsCount=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushOutputSize_min=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_mean=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_min=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:shortCircuitBytesRead=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_99.9th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:DeleteBatch_99th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_median=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_min=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoMissCount=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_99th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileIndexSize=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_99.9th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_mean=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheTrailerHitCount=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_98th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:flushedCellsSize=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCountPrimary=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Put_min=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_max=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,566 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_99.9th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_90th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_90th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_num_ops=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_max=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_mean=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_max=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_min=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:totalRowActionRequestCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:rpcMutateRequestCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_25th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationBytes=108901 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_mean=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheDataHitCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:zeroCopyBytesRead=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_98th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:regionCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_num_ops=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_num_ops=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_median=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mutationsWithoutWALSize=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_num_ops=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_mean=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_median=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:l2CacheFreeSize=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:averageRegionSize=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Get_95th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaMissCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_75th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:filteredReadRequestCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheMetaHitCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_75th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionInputSize_num_ops=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_95th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputSize_median=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheHitCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_num_ops=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:tag.serverName=8c5d72df62de,40585,1697385851811 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReads=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:slowAppendCount=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:numReferenceFiles=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:cellsCountCompactedFromMob=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:storeFileSize=0 2023-10-15T16:24:17,567 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Replay_max=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionTime_90th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_median=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:SplitTime_mean=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_98th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:hedgedReadWins=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:pauseWarnThresholdExceeded=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:AppendBlockBytesScanned_max=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Delete_25th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFileInfoHitCount=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionTime_median=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:memStoreHeapSize=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Bulkload_25th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_min=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_99.9th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanSize_25th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_num_ops=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Append_90th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_max=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:FlushTime_99.9th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_max=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CompactionOutputSize_median=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobScanCellsSize=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_median=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanTime_median=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PutBatch_min=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheBloomChunkMissCount=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:Increment_95th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:blockCacheFreeSize=922070024 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:mobFileCacheHitPercent=0.0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:17,568 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-1:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsCount=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_98th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_median=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestRatePerSecond=0.0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerMissCount=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_mean=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_median=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_98th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_95th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_median=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_90th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_5min_rate=0.0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_min=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_95th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitRatio=0.0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_75th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_90th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_mean=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_90th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_num_ops=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_min=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_25th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_num_ops=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:writeRequestCount=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedOutputBytes=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_90th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkHitCount=1 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadOpsInCurThread=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_median=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestRatePerSecond=0.0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_25th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_min=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_num_ops=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_num_ops=11 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_90th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_98th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockBytesScannedCount=6840 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_max=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_75th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCountPrimary=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_max=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_mean=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_num_ops=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_25th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_75th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheEvictionCount=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99.9th_percentile=0 2023-10-15T16:24:17,569 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheCount=4 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:smallCompactionQueueLength=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheEvictionCount=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_95th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_min=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_num_ops=7 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_num_ops=1 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_mean=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowIncrementCount=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushQueueLength=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_mean=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_75th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:localBytesRead=123986 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushCount=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_median=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutateFailedCount=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_max=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorPoolAllocationBytes=599040 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_98th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_median=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_min=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_median=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_min=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_mean=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:updatesBlockedTime=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_median=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_mean=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitQueueLength=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_min=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_min=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99.9th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_mean=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_95th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCachingCount=4 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_25th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99.9th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileAge=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_median=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_75th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_max=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_95th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_num_ops=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_min=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cpRequestCount=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_median=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_90th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_99th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_median=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_95th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_max=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_num_ops=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALCount=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_98th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitRatio=0.72 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataMissCount=6 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:readRequestCount=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissCount=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_75th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_mean=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheEvictionCount=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheFreeSize=922061632 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_95th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_max=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_median=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_75th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_num_ops=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_max=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_98th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_median=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:activeScanners=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterNegativeResultsCount=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissCount=7 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_mean=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_90th_percentile=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomMissCount=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_max=0 2023-10-15T16:24:17,570 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_max=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_95th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_98th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactionQueueLength=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_95th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_75th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_num_ops=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_mean=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_max=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_25th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_1min_rate=0.0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_min=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_median=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_num_ops=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_median=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_mean=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_75th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_min=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDeleteFamilyBloomHitCount=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99.9th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_median=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_25th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_25th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_99.9th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_90th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_num_ops=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheSize=685248 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99.9th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_num_ops=7 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_min=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_num_ops=11 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_98th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_max=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_count=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_25th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_median=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowGetCount=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_median=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterRequestsCount=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_min=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_min=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_90th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_25th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSizeGrowthRate=0.0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_num_ops=1 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_25th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_max=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileSize=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_min=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:maxStoreFileCount=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsCount=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_99.9th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_98th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_25th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_num_ops=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedOutputBytes=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_75th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99.9th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_max=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_num_ops=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_max=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticBloomSize=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexMissCount=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseInfoThresholdExceeded=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_95th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCount=4 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCount=7 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_max=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedToMob=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedMemstoreBytes=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheExpressHitPercent=81.81818181818183 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionServerStartTime=1697385888915 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedInputBytes=18105 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsSizeCompactedFromMob=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_max=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRequestCount=21 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_25th_percentile=0 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcGetRequestCount=1 2023-10-15T16:24:17,571 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_90th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheCountHitPercent=72.0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_75th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_max=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_mean=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_min=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_98th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexMissCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_min=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedCellsSize=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_mean=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_75th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowDeleteCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockedRequestCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_mean=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheAccessCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_min=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_median=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_min=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_90th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_99th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_min=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_25th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_95th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_99th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_90th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_num_ops=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheEvictedCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_90th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_max=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheIntermediateIndexHitCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:avgStoreFileAge=12090 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_25th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitRequestCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_num_ops=1 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_min=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalBytesRead=123986 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedToMob=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_max=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_25th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_max=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_95th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedInputBytes=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheSize=685248 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_98th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_99th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hlogFileCount=699 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_mean=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcScanRequestCount=11 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreOffHeapSize=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_75th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_98th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_95th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_mean=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheMissCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexMissCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowPutCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_98th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_max=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_75th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreSize=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_75th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsSize=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_mean=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_min=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_75th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_median=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_98th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_median=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Context=regionserver 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99.9th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:scannerLeaseExpiredCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:minStoreFileAge=2959 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheHitCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_98th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcFullScanRequestCount=0 2023-10-15T16:24:17,572 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_90th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_75th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_mean=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:splitSuccessCount=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_mean=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_min=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_75th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_25th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMissCountPrimary=7 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_max=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataBlockCount=3 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_98th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_max=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_median=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsCount=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_mean=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsCount=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCachingCount=18 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_90th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_75th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFailedInsertionCount=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_90th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFlushedCellsSize=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_max=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_mean_rate=0.0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:staticIndexSize=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorUsedBufferCount=1 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_95th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_min=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_90th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_99.9th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheMissRatio=0.28 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_75th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_max=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_mean=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_num_ops=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_95th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_25th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMultiRequestCount=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheSize=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheRootIndexHitCount=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_max=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_98th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:compactedOutputBytes=5921 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocal=0.0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.zookeeperQuorum=127.0.0.1:52753 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_99.9th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_98th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndPut_max=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_num_ops=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_num_ops=1 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_99.9th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_25th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:percentFilesLocalSecondaryRegions=0.0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_min=0 2023-10-15T16:24:17,573 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_mean=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_15min_rate=0.0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_25th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_99.9th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_99th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_min=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_99th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_95th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_75th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_median=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_median=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_mean=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_95th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l1CacheHitCount=18 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_95th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:largeCompactionQueueLength=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_mean=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_99.9th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_median=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:checkMutatePassedCount=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_max=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_25th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_95th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.clusterId=b3f2b9ba-d33e-407e-b223-f02b5843aee1 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_90th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_75th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheMissRatio=0.0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:majorCompactedCellsCount=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_25th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushOutputSize_min=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_mean=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_75th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_min=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:shortCircuitBytesRead=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_75th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_99.9th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:DeleteBatch_99th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndDelete_95th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_90th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_median=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_90th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_98th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_min=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_mean=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_min=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoMissCount=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_90th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_99th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileIndexSize=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_99.9th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_98th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_95th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_mean=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheTrailerHitCount=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_98th_percentile=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:flushedCellsSize=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCountPrimary=18 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Put_min=0 2023-10-15T16:24:17,574 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_max=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_99.9th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_90th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_95th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_90th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_90th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_num_ops=1 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_num_ops=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_max=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_mean=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_98th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_max=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_min=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:totalRowActionRequestCount=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:rpcMutateRequestCount=8 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheCount=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_25th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_99.9th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationBytes=111031 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99.9th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_mean=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_median=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheDataHitCount=17 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:zeroCopyBytesRead=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_98th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:regionCount=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_num_ops=1 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileCount=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_num_ops=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_median=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mutationsWithoutWALSize=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_num_ops=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_95th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_mean=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_median=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:l2CacheFreeSize=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:averageRegionSize=32082 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Get_95th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaMissCount=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_mean=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_98th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_75th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:filteredReadRequestCount=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheMetaHitCount=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushMemstoreSize_99th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:GetBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_98th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_75th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheLeafIndexHitCount=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionInputSize_num_ops=1 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_95th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_num_ops=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_99th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputSize_median=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheHitCount=18 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_num_ops=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:tag.serverName=8c5d72df62de,35419,1697385888915 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReads=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputFileCount_99.9th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionInputFileCount_99th_percentile=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_min=0 2023-10-15T16:24:17,575 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:slowAppendCount=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:numReferenceFiles=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:cellsCountCompactedFromMob=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:storeFileSize=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Replay_max=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_99.9th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_mean=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorHeapAllocationRatio=0.0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionTime_90th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_median=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:SplitTime_mean=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_98th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:hedgedReadWins=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:pauseWarnThresholdExceeded=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_90th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutateBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:AppendBlockBytesScanned_max=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Delete_25th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFileInfoHitCount=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionTime_median=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:memStoreHeapSize=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Bulkload_25th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_min=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_99.9th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanSize_25th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_num_ops=1 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ByteBuffAllocatorTotalBufferCount=186 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Append_90th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_max=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:FlushTime_99.9th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_95th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_max=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputFileCount_98th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CompactionOutputSize_median=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaHitCount=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobScanCellsSize=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_median=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:IncrementBlockBytesScanned_75th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:CheckAndMutate_99th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanTime_median=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:MajorCompactionOutputSize_25th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PutBatch_min=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheBloomChunkMissCount=1 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99.9th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheGeneralBloomMetaMissCount=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:Increment_95th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:blockCacheFreeSize=922061632 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:ScanBlockBytesScanned_99th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:mobFileCacheHitPercent=0.0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithoutGc_25th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:bloomFilterEligibleRequestsCount=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Server-3:PauseTimeWithGc_num_ops=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_98th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_num_ops=2 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.walReaderEditsBufferUsage=80117 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_25th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfHFileRefsQueue=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logReadInBytes=83847 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_75th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.sizeOfLogQueue=-3 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_90th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_99.9th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_num_ops=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedOps=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_min=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedHFiles=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_min=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_mean=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBytes=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_90th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedOps=1000 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedBatches=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_98th_percentile=0 2023-10-15T16:24:17,576 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_max=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_95th_percentile=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.appliedBatches=1 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_mean=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_95th_percentile=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedHFiles=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_median=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.uncleanlyClosedLogs=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99th_percentile=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.closedLogsWithUnknownFileLength=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_max=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.repeatedLogFileBytes=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedRecoverQueues=5 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Context=regionserver 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.failedRecoverQueues=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.restartedLogReading=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.ignoredUncleanlyClosedLogContentsInBytes=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsRead=40 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.numInitializing=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Source.ageOfLastShippedOp_99.9th_percentile=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_75th_percentile=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.logEditsFiltered=38 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_25th_percentile=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.completedLogs=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:sink.failedBatches=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:source.shippedBatches=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Replication:Sink.ageOfLastAppliedOp_median=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsAvgTime=0.0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsAvgTime=0.0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpNumOps=1 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWriteFallback=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpNumOps=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksWrite=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersisted=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosNumOps=26 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksDeletedInPendingIBR=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRemoved=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalAvgTime=1.0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CacheReportsNumOps=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksCached=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsAvgTime=94.0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DataNodeActiveXceiversCount=-2 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsTotalNumOps=9 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpAvgTime=0.0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsNumOps=32 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsAvgTime=0.0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesLazyPersisted=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesRead=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ProcessedCommandsOpAvgTime=1.0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksLazyPersistWindowMsNumOps=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpAvgTime=6326.928571428572 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesAvgTime=0.0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionTasks=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncCount=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:NumProcessedCommands=10 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpNumOps=27 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromLocalClient=28 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:VolumeFailures=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcInvalidReconstructionTasks=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosNumOps=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesRead=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksReadHits=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:PacketAckRoundTripTimeNanosAvgTime=561936.0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalReadTime=9 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesRead=52810 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:IncrementalBlockReportsAvgTime=1.7142857142857142 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromRemoteClient=0 2023-10-15T16:24:17,577 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictedWithoutRead=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksUncached=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:DatanodeNetworkErrors=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadsFromLocalClient=27 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksInPendingIBR=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockVerificationFailures=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpAvgTime=0.0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosNumOps=54 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBytesWrite=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosNumOps=54 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.SessionId=null 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockReportsNumOps=1 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivedInPendingIBR=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionDecodingTimeMillis=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvictionWindowMsNumOps=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BytesWritten=620489 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionRemoteBytesRead=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcFailedReconstructionTasks=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksRead=27 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReplicated=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Context=dfs 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksVerified=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WritesFromRemoteClient=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksGetLocalPathInfo=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketTransferNanosAvgTime=20303.65 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksDeletedBeforeLazyPersisted=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlockChecksumOpNumOps=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsNumOps=9 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksWritten=28 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:HeartbeatsAvgTime=1.0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SumOfActorCommandQueueLength=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:TotalWriteTime=12 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RamDiskBlocksEvicted=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcDecodingTimeNanos=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionReadTimeMillis=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:LifelinesNumOps=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosNumOps=1285 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:WriteBlockOpNumOps=28 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FlushNanosAvgTime=23635.714285714286 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReplaceBlockOpAvgTime=0.0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:RemoteBytesWritten=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:FsyncNanosAvgTime=0.0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionBytesWritten=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:SendDataPacketBlockedOnNetworkNanosAvgTime=10588.8 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:ReadBlockOpAvgTime=1.0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:CopyBlockOpNumOps=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:EcReconstructionWriteTimeMillis=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeActivity-127.0.0.1-39187:BlocksReceivingInPendingIBR=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumFailedVolumes=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheCapacity=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:CacheUsed=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:FailedStorageLocations=[Ljava.lang.String;@437a5445 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksCached=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:LastVolumeFailureDate=0 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Capacity=1054589059072 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4]'} 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToUncache=10 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:DfsUsed=1039261 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:Remaining=941144442277 2023-10-15T16:24:17,578 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-d111daf0-5bdd-4f4e-87d2-aab5b448b3d9:NumBlocksFailedToCache=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numRegionSizeReports=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_min=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_num_ops=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_98th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_num_ops=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numNamespaceInQuotaViolation=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_75th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_95th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_90th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_median=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_max=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.namespaceSpaceQuotaOverview=[] 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numTablesInQuotaViolation=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_num_ops=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99.9th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_max=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_95th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_98th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_98th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_99.9th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_median=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_90th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_99th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_min=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_max=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_75th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Context=master 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_99.9th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_95th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_median=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_mean=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_mean=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_max=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_25th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_mean=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_min=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_75th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_mean=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_90th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_min=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSizeComputationTime_median=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:tag.tableSpaceQuotaOverview=[] 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:numSpaceQuotas=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:QuotaObserverChoreTime_25th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotObserverSnapshotFetchTime_25th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=Quotas:SnapshotQuotaObserverChoreTime_num_ops=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenSubmittedCount=14 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignSubmittedCount=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_num_ops=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99.9th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_median=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_max=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_95th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_95th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_min=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_mean=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_min=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_median=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_min=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveSubmittedCount=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_25th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_mean=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_99.9th_percentile=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_max=0 2023-10-15T16:24:17,579 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnFS=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerRegions=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_75th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_95th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignSubmittedCount=13 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_75th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_98th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_num_ops=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_90th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignFailedCount=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_num_ops=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_90th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignFailedCount=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_25th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_98th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_98th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveFailedCount=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_mean=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_75th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeFailedCount=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Context=master 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_num_ops=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseSubmittedCount=1 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:holes=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_75th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99.9th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_95th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_95th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeSubmittedCount=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99.9th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_median=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_99.9th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_mean=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCount=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenSubmittedCount=1 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseFailedCount=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_99th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_max=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:inconsistentRegions=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_90th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenFailedCount=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:orphanRegionsOnRS=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_99.9th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_75th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_75th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_min=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_25th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:unknownServerOpenRegions=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_95th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_25th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:overlaps=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenFailedCount=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_90th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_98th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitFailedCount=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_max=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_98th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_mean=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_num_ops=13 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_median=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_mean=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritCountOverThreshold=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_min=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_75th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_90th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_num_ops=1 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_75th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_95th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_25th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_median=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_25th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitSubmittedCount=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_mean=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ritOldestAge=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:emptyRegionInfoRegions=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_max=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_25th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_99.9th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_90th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_min=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_99.9th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_num_ops=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_max=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_95th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_median=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:SplitTime_max=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_90th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_25th_percentile=0 2023-10-15T16:24:17,580 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_25th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_min=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_90th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_95th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_max=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_99th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_num_ops=1 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_min=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:operationCount=15 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_98th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:deadServerOpenRegions=3 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_mean=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:RitDuration_99th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_median=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MergeTime_min=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_99.9th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:ReopenTime_98th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:CloseTime_median=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_max=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_mean=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_num_ops=14 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_98th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:AssignTime_90th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:MoveTime_median=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:OpenTime_98th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): Master,sub=AssignmentManager:UnassignTime_75th_percentile=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumFailedVolumes=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheUsed=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:CacheCapacity=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksCached=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:LastVolumeFailureDate=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Capacity=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Context=FSDatasetState 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/f4379044-1cd9-a06c-d54b-12cfb419c71e/cluster_412979e8-fcf0-f45e-7288-e58551c8e6b2/dfs/data/data2]'} 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:DfsUsed=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:Remaining=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToCache=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-1:NumBlocksFailedToUnCache=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalDataFileIos=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalFileIoErrors=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateNumOps=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateNumOps=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:DataFileIoRateNumOps=0 2023-10-15T16:24:17,581 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateNumOps=0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateNumOps=0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:FlushIoRateNumOps=0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Context=dfs 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TotalMetadataOperations=0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsNumOps=17 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportNumOps=14 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsNumOps=57 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetServerDefaultsAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetFileInfoNumOps=707 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestNumOps=3 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReceivedAndDeletedNumOps=172 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateNumOps=64 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.port=39027 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeNumOps=3 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportAvgTime=1.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Context=rpcdetailed 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenameNumOps=44 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetDatanodeReportAvgTime=0.21428571428571427 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteAvgTime=0.25 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateAvgTime=5.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteNumOps=64 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingNumOps=219 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetBlockLocationsNumOps=334 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:BlockReportNumOps=4 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:GetListingAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CompleteAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:VersionRequestAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RegisterDatanodeAvgTime=0.6666666666666666 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:CreateAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetPermissionNumOps=1 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:AddBlockNumOps=53 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:DeleteNumOps=41 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:TruncateNumOps=1 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetTimesNumOps=16 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:MkdirsAvgTime=0.0 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SetSafeModeNumOps=1 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatNumOps=1200 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RenewLeaseNumOps=39 2023-10-15T16:24:17,582 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseNumOps=5 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:RecoverLeaseAvgTime=0.0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcDetailedActivityForPort39027:SendHeartbeatAvgTime=0.0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemMaxM=2200.0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTerminated=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsWaiting=87 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.SessionId=null 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Context=jvm 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapUsedM=193.45564 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTotalExtraSleepTime=55 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Old Generation=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCount=11 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsBlocked=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogWarn=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsRunnable=84 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogInfo=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsNew=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapUsedM=566.47656 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:ThreadsTimedWaiting=118 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:MemHeapMaxM=2200.0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcCountG1 Young Generation=11 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogError=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:tag.ProcessName=DataNode 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:LogFatal=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcTimeMillis=328 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-3:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDataNodeMessageCount=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastWrittenTransactionId=466 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:SnapshottableDirectories=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEncryptionZones=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReconstructionBlocks=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptBlocks=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LastCheckpointTime=1697385856136 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ReadLockLongHoldCount=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumActiveClients=1 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastLogRoll=466 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumTimedOutPendingReconstructions=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:VolumeFailuresTotal=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ProvidedCapacityTotal=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingECBlockGroups=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedGB=0.0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionReplicatedBlocks=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecommissioningDataNodes=0 2023-10-15T16:24:17,583 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.HAState=active 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyECBlockGroups=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:FilesTotal=109 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ScheduledReplicationBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureECBlockGroups=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptECBlockGroups=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MillisSinceLastLoadedEdits=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalECBlockGroups=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExcessBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalLoad=9 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:ExpiredHeartbeats=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumInServiceLiveDataNodes=3 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumStaleStorages=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:WriteLockLongHoldCount=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlocksTotal=43 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:UnderReplicatedBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicatedBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsed=3126472 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalReplicatedBlocks=43 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:StaleDataNodes=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumFilesUnderConstruction=1 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemaining=2823484494063 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PostponedMisreplicatedBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityUsedNonDFS=178715290424 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LockQueueLength=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CorruptReplicatedBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BytesInFutureReplicatedBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Context=dfs 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomDeadDataNodes=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:Snapshots=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingDeletionECBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.TotalSyncTimes=11 5 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:PendingReplicationBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotal=3163767177216 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDeadDataNodes=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplOneBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TotalSyncCount=417 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:TransactionsSinceLastCheckpoint=466 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:MissingReplicationOneBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumLiveDataNodes=3 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:BlockCapacity=4194304 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:tag.EnabledEcPolicies=RS-6-3-1024k 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityTotalGB=2946.0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:LowRedundancyBlocks=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:NumDecomLiveDataNodes=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystem-1:CapacityRemainingGB=2630.0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:ReceivedBytes=15456 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeAvgTime=0.0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcSlowCalls=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationSuccesses=46 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumDroppedConnections=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.NumOpenConnectionsPerUser={} 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcClientBackoff=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeAvgTime=0.0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthorizationFailures=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:NumOpenConnections=0 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.port=35085 2023-10-15T16:24:17,584 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:SentBytes=9822 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcProcessingTimeNumOps=46 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationSuccesses=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:CallQueueLength=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Context=rpc 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeAvgTime=0.0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcAuthenticationFailures=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcLockWaitTimeNumOps=46 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.serverName=ClientDatanodeProtocolService 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:RpcQueueTimeNumOps=46 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RpcActivityForPort35085:DeferredRpcProcessingTimeNumOps=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:receivedBytes=288613 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_98th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numGeneralCallsDropped=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationSuccesses=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyTotalPendingOutboundBytes=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_mean=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_25th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_75th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReplicationHandler=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_25th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_95th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_max=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callQueueTooBig=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveWriteHandler=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_98th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_90th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_90th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFailures=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_98th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_75th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numOpenConnections=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_75th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_min=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_num_ops=1195 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:maxOutboundBytesExceeded=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99.9th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callTimedOut=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_median=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_90th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReplicationQueue=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationSuccesses=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_25th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_num_ops=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99.9th_percentile=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_max=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_num_ops=1195 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_num_ops=1195 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_max=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInMetaPriorityQueue=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInReadQueue=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_max=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_median=0 2023-10-15T16:24:17,585 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInWriteQueue=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.OutOfOrderScannerNextException=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.rpcThrottling=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.otherExceptions=5 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInScanQueue=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_95th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Context=regionserver 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_75th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_median=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_min=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInGeneralQueue=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.FailedSanityCheckException=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99.9th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_25th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_median=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_max=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_median=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_95th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_min=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveBulkLoadHandler=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_98th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_mean=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_min=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_99.9th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_99.9th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInBulkLoadQueue=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_99th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveHandler=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.requestTooBig=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveGeneralHandler=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_num_ops=1195 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_mean=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveReadHandler=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_median=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_90th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_num_ops=1195 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActivePriorityHandler=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_99th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_mean=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_95th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_25th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.callDropped=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_75th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numCallsInPriorityQueue=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions=5 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authenticationFallbacks=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.quotaExceeded=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyDirectMemoryUsage=67108864 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_99th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionMovedException=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:authorizationFailures=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_min=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_90th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_25th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_95th_percentile=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numLifoModeSwitches=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_max=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:queueSize=0 2023-10-15T16:24:17,586 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.multiResponseTooLarge=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99.9th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.RegionTooBusyException=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.UnknownScannerException=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:numActiveScanHandler=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_95th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_98th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_75th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:TotalCallTime_98th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ResponseSize_90th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:RequestSize_99th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.NotServingRegionException=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:QueueCallTime_min=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:ProcessCallTime_mean=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:UnwritableTime_mean=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:nettyMaxPendingOutboundBytes=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:exceptions.ScannerResetException=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=IPC:sentBytes=44933 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_mean=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99.9th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_75th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_max=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_90th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_min=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_median=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_num_ops=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOnHeapSize=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_num_ops=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_num_ops=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_median=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_num_ops=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_75th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_min=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreOffHeapSize=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_25th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_75th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_mean=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_25th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_25th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_mean=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tunerDoNothingCounter=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_25th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_num_ops=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:aboveHeapOccupancyLowWaterMarkCounter=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_98th_percentile=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:memStoreSize=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_min=0 2023-10-15T16:24:17,587 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_max=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_90th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_mean=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_75th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_99.9th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Context=regionserver 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99.9th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_mean=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_min=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockCacheSize=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_90th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:blockedFlushGauge=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_max=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_median=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_98th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_99th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:unblockedFlushGauge=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_max=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_max=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_75th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_98th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_num_ops=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_min=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_95th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_90th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_90th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_95th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_max=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_95th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_median=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_98th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseBlockCacheSize_min=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_25th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99.9th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_median=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseBlockCacheSize_median=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:BlockedFlushes_99th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_99th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_95th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:IncreaseMemStoreSize_mean=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:DecreaseMemStoreSize_95th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): RegionServer,sub=Memory:UnblockedFlushes_98th_percentile=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalDataFileIos=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalFileIoErrors=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateNumOps=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateNumOps=0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,588 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:DataFileIoRateNumOps=0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateNumOps=0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateNumOps=0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:FlushIoRateNumOps=0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Context=dfs 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TotalMetadataOperations=0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:HttpPort=null 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XceiverCount=3 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SendPacketDownstreamAvgInfo=null 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:BPServiceActorInfo=[{"maxDataLength":"67108864","LastHeartbeat":"0","ActorState":"RUNNING","NamenodeAddress":"localhost:39027","BlockPoolID":"BP-1531809096-172.17.0.3-1697385856100","maxBlockReportSize":"0","LastBlockReport":"1200"}] 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:ClusterId=testClusterID 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:VolumeInfo={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3":{"numBlocks":21,"usedSpace":259482,"freeSpace":470504981925,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":134216283},"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4":{"numBlocks":22,"usedSpace":779779,"freeSpace":470639198208,"reservedSpace":0,"storageType":"DISK","reservedSpaceForReplicas":0}} 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DatanodeHostname=127.0.0.1 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SecurityEnabled=false 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DiskBalancerStatus={"currentState":[],"result":"NO_PLAN","planID":"","planFile":""} 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:RpcPort=0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DataPort=0 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SoftwareVersion=3.2.4 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:17,589 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:SlowDisks=null 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:XmitsInProgress=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:DNStartedTimeInMillis=1697385857021 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeInfo-4:NamenodeAddresses={"localhost":"BP-1531809096-172.17.0.3-1697385856100"} 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsBatchedInSync=50 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsNumOps=416 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SyncsAvgTime=0.0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportNumOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeNumOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteSnapshotOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsAvgTime=0.05714285714285714 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AllowSnapshotOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:RenameSnapshotOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeAvgTime=0.05 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesCreated=176 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesAppended=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CacheReportAvgTime=0.0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalAvgTime=0.0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeAvgTime=0.0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageAvgTime=0.0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeNumOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogFetchTimeNumOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ResourceCheckTimeNumOps=246 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailTimeAvgTime=0.0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumTimesReReplicationNotScheduled=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SafeModeTime=1 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:EditLogTailIntervalNumOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetListingOps=138 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.ProcessName=NameNode 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetImageNumOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TotalFileOps=1372 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSymlinkOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:AddBlockOps=53 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DisallowSnapshotOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:DeleteFileOps=31 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TransactionsNumOps=466 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateSnapshotOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PendingEditsCount=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:ListSnapshottableDirOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockReceivedAndDeletedOps=172 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.SessionId=null 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeAvgTime=0.0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditNumOps=0 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FsImageLoadTime=23 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:CreateFileOps=64 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Context=dfs 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesRenamed=44 2023-10-15T16:24:17,590 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedAvgCount=0.0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:TimeoutReReplications=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsQueued=1 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:NumEditLogLoadedNumOps=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageNumOps=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:PutImageAvgTime=0.0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesInGetListingOps=154 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FileInfoOps=707 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetLinkTargetOps=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetAdditionalDatanodeOps=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:BlockOpsBatched=71 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesDeleted=47 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SuccessfulReReplications=23 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetBlockLocations=334 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:FilesTruncated=1 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:SnapshotDiffReportOps=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportNumOps=8 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GenerateEDEKTimeNumOps=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:GetEditAvgTime=0.0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:StorageBlockReportAvgTime=0.5 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeActivity-1:WarmUpEDEKTimeAvgTime=0.0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumFailedVolumes=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheUsed=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:CacheCapacity=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksCached=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:LastVolumeFailureDate=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Capacity=1054589059072 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Context=FSDatasetState 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.StorageInfo=FSDataset{dirpath='[/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1, /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2]'} 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:DfsUsed=1039261 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:Remaining=941156091301 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToCache=0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSDatasetState-3:NumBlocksFailedToUnCache=10 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateAvgTime=0.0 2023-10-15T16:24:17,591 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateAvgTime=0.0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalDataFileIos=0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:MetadataOperationRateNumOps=0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalFileIoErrors=0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateNumOps=0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:NativeCopyIoRateNumOps=0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateAvgTime=0.0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateAvgTime=0.0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateNumOps=0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateAvgTime=0.0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:DataFileIoRateNumOps=0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:ReadIoRateAvgTime=0.0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FileIoErrorRateNumOps=0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateNumOps=0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:WriteIoRateAvgTime=0.0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateNumOps=0 2023-10-15T16:24:17,592 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:FlushIoRateNumOps=0 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Context=dfs 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TotalMetadataOperations=0 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:SyncIoRateAvgTime=0.0 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:TransferIoRateAvgTime=0.0 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): DataNodeVolume-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessNumOps=1 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureNumOps=0 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailures=0 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsNumOps=4 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:RenewalFailuresTotal=0 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Context=ugi 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:GetGroupsAvgTime=29.0 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginSuccessAvgTime=10.0 2023-10-15T16:24:17,593 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): UgiMetrics:LoginFailureAvgTime=0.0 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfSnapshottableDirs=0 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheCapacity=0 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CacheUsed=0 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ProvidedCapacity=0 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyECBlocks=0 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DistinctVersionCount=1 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocksWithReplicationFactorOne=0 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentRemaining=89.24353 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Version=3.2.4, r7e5d9983b388e372fe640f21f048f2f2ae6e9eba 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentUsed=9.8821176E-5 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:TotalBlocks=43 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NonDfsUsedSpace=178742053688 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Safemode= 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:UpgradeFinalized=true 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:EnteringMaintenanceNodes={} 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:PercentBlockPoolUsed=9.8821176E-5 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:LiveNodes={"127.0.0.1:40557":{"infoAddr":"127.0.0.1:35357","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:40557","lastContact":0,"usedSpace":1039261,"adminState":"In Service","nonDfsUsedSpace":59586397283,"capacity":1054589059072,"numBlocks":42,"version":"3.2.4","used":1039261,"remaining":941146867109,"blockScheduled":1,"blockPoolUsed":1039261,"blockPoolUsedPercent":9.854653E-5,"volfails":0,"lastBlockReport":20},"127.0.0.1:46365":{"infoAddr":"127.0.0.1:35919","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46365","lastContact":0,"usedSpace":1039261,"adminState":"In Service","nonDfsUsedSpace":59583407203,"capacity":1054589059072,"numBlocks":42,"version":"3.2.4","used":1039261,"remaining":941149857189,"blockScheduled":1,"blockPoolUsed":1039261,"blockPoolUsedPercent":9.854653E-5,"volfails":0,"lastBlockReport":8},"127.0.0.1:46399":{"infoAddr":"127.0.0.1:44887","infoSecureAddr":"127.0.0.1:0","xferaddr":"127.0.0.1:46399","lastContact":1,"usedSpace":1047950,"adminState":"In Service","nonDfsUsedSpace":59572249202,"capacity":1054589059072,"numBlocks":42,"version":"3.2.4","used":1047950,"remaining":941161006501,"blockScheduled":1,"blockPoolUsed":1047950,"blockPoolUsedPercent":9.937046E-5,"volfails":0,"lastBlockReport":19}} 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CompileInfo=2022-07-12T11:58Z by ubuntu from branch-3.2.4 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolId=BP-1531809096-172.17.0.3-1697385856100 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:ClusterId=testClusterID 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DeadNodes={} 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:JournalTransactionInfo={"MostRecentCheckpointTxId":"0","LastAppliedOrWrittenTxId":"466"} 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Free=2823457730799 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:BlockPoolUsedSpace=3126472 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Threads=289 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:HighestPriorityLowRedundancyReplicatedBlocks=0 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NumberOfMissingBlocks=0 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirStatuses={"active":{"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":"IMAGE_AND_EDITS","/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":"IMAGE_AND_EDITS"},"failed":{}} 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:SoftwareVersion=3.2.4 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:VerifyECWithTopologyResult={"isSupported":"false","resultMessage":"The number of DataNodes (3) is less than the minimum required number of DataNodes (9) for the erasure coding policies: RS-6-3-1024k"} 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Used=3126472 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:Total=3163767177216 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NodeUsage={"nodeUsage":{"min":"0.00%","median":"0.00%","max":"0.00%","stdDev":"0.00%"}} 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NNStartedTimeInMillis=1697385856352 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:DecomNodes={} 2023-10-15T16:24:17,594 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:CorruptFiles=[] 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameDirSize={"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1":737,"/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2":737} 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): NameNodeInfo-1:NameJournalStatus=[{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-1/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"},{"manager":"FileJournalManager(root=/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2)","stream":"EditLogFileOutputStream(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/name-0-2/current/edits_inprogress_0000000000000000001)","disabled":"false","required":"false"}] 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Old Generation=0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemMaxM=2200.0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillisG1 Young Generation=328 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTerminated=0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsWaiting=87 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.SessionId=null 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Context=jvm 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapCommittedM=197.72656 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapUsedM=193.46402 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemNonHeapMaxM=-1.0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTotalExtraSleepTime=55 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.Hostname=8c5d72df62de 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Old Generation=0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCount=11 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsBlocked=0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogWarn=0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapCommittedM=1000.0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsRunnable=84 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumWarnThresholdExceeded=0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogInfo=0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsNew=0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapUsedM=567.47656 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:ThreadsTimedWaiting=118 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:MemHeapMaxM=2200.0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcCountG1 Young Generation=11 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogError=0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:tag.ProcessName=DataNode 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:LogFatal=0 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcTimeMillis=328 2023-10-15T16:24:17,595 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): JvmMetrics-8:GcNumInfoThresholdExceeded=0 2023-10-15T16:24:17,596 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlocksTotal=43 2023-10-15T16:24:17,596 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TopUserOpCounts={"timestamp":"2023-10-15T16:24:17+0000","windows":[{"ops":[{"opType":"*","totalCount":0,"allUsers":[],"topUsers":[]}],"windowLenMs":60000},{"ops":[{"opType":"*","totalCount":53,"allUsers":[{"user":"jenkins","count":53}],"topUsers":[{"user":"jenkins","count":53}]},{"opType":"getfileinfo","totalCount":31,"allUsers":[{"user":"jenkins","count":31}],"topUsers":[{"user":"jenkins","count":31}]},{"opType":"rename","totalCount":3,"allUsers":[{"user":"jenkins","count":3}],"topUsers":[{"user":"jenkins","count":3}]},{"opType":"mkdirs","totalCount":3,"allUsers":[{"user":"jenkins","count":3}],"topUsers":[{"user":"jenkins","count":3}]},{"opType":"create","totalCount":4,"allUsers":[{"user":"jenkins","count":4}],"topUsers":[{"user":"jenkins","count":4}]},{"opType":"open","totalCount":12,"allUsers":[{"user":"jenkins","count":12}],"topUsers":[{"user":"jenkins","count":12}]}],"windowLenMs":300000},{"ops":[{"opType":"*","totalCount":1499,"allUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":375},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":512},{"user":"jenkins","count":375},{"user":"jenkins.hfs.6","count":258},{"user":"jenkins.hfs.5","count":198},{"user":"jenkins.hfs.3","count":155},{"user":"jenkins.hfs.2","count":1}]},{"opType":"datanodeReport","totalCount":14,"allUsers":[{"user":"jenkins","count":14}],"topUsers":[{"user":"jenkins","count":14}]},{"opType":"listStatus","totalCount":219,"allUsers":[{"user":"jenkins","count":79},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}],"topUsers":[{"user":"jenkins","count":79},{"user":"jenkins.hfs.4","count":47},{"user":"jenkins.hfs.6","count":39},{"user":"jenkins.hfs.5","count":38},{"user":"jenkins.hfs.3","count":16}]},{"opType":"safemode_get","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"delete","totalCount":41,"allUsers":[{"user":"jenkins","count":20},{"user":"jenkins.hfs.4","count":13},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}],"topUsers":[{"user":"jenkins","count":20},{"user":"jenkins.hfs.4","count":13},{"user":"jenkins.hfs.5","count":3},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.3","count":2}]},{"opType":"getfileinfo","totalCount":707,"allUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":168},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}],"topUsers":[{"user":"jenkins.hfs.4","count":259},{"user":"jenkins","count":168},{"user":"jenkins.hfs.6","count":122},{"user":"jenkins.hfs.5","count":89},{"user":"jenkins.hfs.3","count":69}]},{"opType":"truncate","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"rename","totalCount":44,"allUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":15},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":17},{"user":"jenkins","count":15},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.5","count":1}]},{"opType":"mkdirs","totalCount":57,"allUsers":[{"user":"jenkins","count":35},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}],"topUsers":[{"user":"jenkins","count":35},{"user":"jenkins.hfs.4","count":10},{"user":"jenkins.hfs.3","count":7},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":2}]},{"opType":"create","totalCount":64,"allUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":21},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}],"topUsers":[{"user":"jenkins.hfs.4","count":23},{"user":"jenkins","count":21},{"user":"jenkins.hfs.3","count":9},{"user":"jenkins.hfs.5","count":6},{"user":"jenkins.hfs.6","count":4},{"user":"jenkins.hfs.2","count":1}]},{"opType":"setPermission","totalCount":1,"allUsers":[{"user":"jenkins","count":1}],"topUsers":[{"user":"jenkins","count":1}]},{"opType":"setTimes","totalCount":16,"allUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}],"topUsers":[{"user":"jenkins.hfs.3","count":4},{"user":"jenkins.hfs.4","count":4},{"user":"jenkins","count":4},{"user":"jenkins.hfs.6","count":3},{"user":"jenkins.hfs.5","count":1}]},{"opType":"open","totalCount":334,"allUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":16}],"topUsers":[{"user":"jenkins.hfs.4","count":139},{"user":"jenkins.hfs.6","count":80},{"user":"jenkins.hfs.5","count":58},{"user":"jenkins.hfs.3","count":41},{"user":"jenkins","count":16}]}],"windowLenMs":1500000}]} 2023-10-15T16:24:17,596 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:SnapshotStats={"SnapshottableDirectories":0,"Snapshots":0} 2023-10-15T16:24:17,596 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:UnderReplicatedBlocks=0 2023-10-15T16:24:17,596 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceLiveDataNodes=0 2023-10-15T16:24:17,596 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityUsed=3126472 2023-10-15T16:24:17,596 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEncryptionZones=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityRemaining=2823457730799 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReconstructionBlocks=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumEnteringMaintenanceDataNodes=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomDeadDataNodes=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FSState=Operational 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:VolumeFailuresTotal=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ProvidedCapacityTotal=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:EstimatedCapacityLostTotal=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecommissioningDataNodes=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingReplicationBlocks=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncTimes=11 5 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:PendingDeletionBlocks=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:CapacityTotal=3163767177216 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FilesTotal=109 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:ScheduledReplicationBlocks=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDeadDataNodes=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalSyncCount=417 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:FsLockQueueLength=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:MaxObjects=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:TotalLoad=9 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleDataNodes=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInMaintenanceDeadDataNodes=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumLiveDataNodes=3 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:BlockDeletionStartTime=1697385856352 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumInServiceLiveDataNodes=3 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:LowRedundancyBlocks=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumDecomLiveDataNodes=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(101): FSNamesystemState-1:NumStaleStorages=0 2023-10-15T16:24:17,597 INFO [pool-270-thread-1 {}] common.MetricsLoggerTask(109): << End DataNode metrics dump 2023-10-15T16:24:18,309 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:18,760 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:24:19,309 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:20,309 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:20,942 WARN [HBase-Metrics2-1 {}] impl.MetricsConfig(136): Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 2023-10-15T16:24:21,310 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:21,600 INFO [ProcExecTimeout {}] assignment.AssignmentManager$DeadServerMetricRegionChore(1502): Found 3 OPEN regions on dead servers and 0 OPEN regions on unknown servers 2023-10-15T16:24:21,989 INFO [master/8c5d72df62de:0.Chore.1 {}] master.HMaster(2059): Not running balancer because processing dead regionserver(s): 8c5d72df62de,38435,1697385859175, 8c5d72df62de,46527,1697385859222, 8c5d72df62de,35419,1697385888915, 8c5d72df62de,45999,1697385859285 2023-10-15T16:24:21,989 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(188): Hosts are {8c5d72df62de=0} racks are {/default-rack=0} 2023-10-15T16:24:21,989 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(287): server 0 has 3 regions 2023-10-15T16:24:21,989 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(294): server 0 is on host 0 2023-10-15T16:24:21,989 INFO [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(305): server 0 is on rack 0 2023-10-15T16:24:21,989 DEBUG [master/8c5d72df62de:0.Chore.1 {}] balancer.BalancerClusterState(310): Number of tables=3, number of hosts=1, number of racks=1 2023-10-15T16:24:21,989 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:21,989 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:21,990 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:24:21,990 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:21,990 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:24:21,990 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:21,991 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:22,100 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:22,100 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:22,100 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:24:22,101 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:22,101 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:22,101 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:22,310 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:22,310 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:22,310 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:22,310 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:24:22,311 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:22,311 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:22,311 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:22,620 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:22,620 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:22,620 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:24:22,621 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:22,621 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:22,621 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:23,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:23,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:23,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:24:23,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:23,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:23,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:23,310 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:24,150 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:24,150 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:24,151 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:24:24,151 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2162 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:24:24,152 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:24,152 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:24:24,152 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:24,152 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:24,310 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:25,310 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:26,180 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:26,180 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:26,181 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:24:26,181 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4192 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:24:26,182 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:26,182 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:24:26,182 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:26,182 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:26,311 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:27,311 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:28,311 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:29,311 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:30,220 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:30,220 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:30,221 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:24:30,221 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8231 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:24:30,222 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:30,222 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:24:30,222 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:30,222 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:30,311 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:31,312 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:32,312 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:33,312 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:34,312 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:34,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:24:34,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:24:35,312 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:35,370 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:35,370 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:35,371 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:24:35,371 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 37, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 551106 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:24:35,372 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:35,372 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:24:35,372 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:35,372 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:36,313 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:37,313 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:38,313 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:39,043 WARN [Abort regionserver monitor {}] regionserver.HRegionServer$SystemExitWhenAbortTimeout(3561): Aborting region server timed out, terminating forcibly and does not wait for any running shutdown hooks or finalizers to finish their work. Thread dump to stdout. Process Thread Dump: Zombie HRegionServer 290 active threads Thread 1 (main): State: TIMED_WAITING Blocked count: 7 Waited count: 5 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.FutureTask.awaitDone(FutureTask.java:444) java.base@11.0.17/java.util.concurrent.FutureTask.get(FutureTask.java:203) app//org.junit.internal.runners.statements.FailOnTimeout.getResult(FailOnTimeout.java:167) app//org.junit.internal.runners.statements.FailOnTimeout.evaluate(FailOnTimeout.java:128) app//org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) app//org.junit.rules.RunRules.evaluate(RunRules.java:20) app//org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) app//org.junit.runners.ParentRunner.run(ParentRunner.java:413) app//org.junit.runners.Suite.runChild(Suite.java:128) app//org.junit.runners.Suite.runChild(Suite.java:27) app//org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) app//org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) app//org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) app//org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) app//org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) app//org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) app//org.junit.runners.ParentRunner.run(ParentRunner.java:413) app//org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:49) Thread 2 (Reference Handler): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: java.base@11.0.17/java.lang.ref.Reference.waitForReferencePendingList(Native Method) java.base@11.0.17/java.lang.ref.Reference.processPendingReferences(Reference.java:241) java.base@11.0.17/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213) Thread 3 (Finalizer): State: WAITING Blocked count: 12 Waited count: 13 Waiting on java.lang.ref.ReferenceQueue$Lock@1edc134a Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176) java.base@11.0.17/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170) Thread 4 (Signal Dispatcher): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: Thread 11 (Common-Cleaner): State: TIMED_WAITING Blocked count: 12 Waited count: 32 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) java.base@11.0.17/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) java.base@11.0.17/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:161) Thread 12 (pool-1-thread-1): State: RUNNABLE Blocked count: 0 Waited count: 62 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollPort$EventHandlerTask.poll(EPollPort.java:200) java.base@11.0.17/sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:281) java.base@11.0.17/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 13 (pool-1-thread-2): State: WAITING Blocked count: 0 Waited count: 66 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@51ef43d2 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:417) java.base@11.0.17/sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:275) java.base@11.0.17/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 14 (surefire-forkedjvm-stream-flusher): State: TIMED_WAITING Blocked count: 0 Waited count: 12349 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 16 (surefire-forkedjvm-command-thread): State: WAITING Blocked count: 0 Waited count: 126 Waiting on java.util.concurrent.CountDownLatch$Sync@2f17fa1c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232) java.base@11.0.17/sun.nio.ch.PendingFuture.get(PendingFuture.java:178) app//org.apache.maven.surefire.api.util.internal.Channels$2.read(Channels.java:127) java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) app//org.apache.maven.surefire.api.util.internal.Channels$3.readImpl(Channels.java:169) app//org.apache.maven.surefire.api.util.internal.AbstractNoninterruptibleReadableChannel.read(AbstractNoninterruptibleReadableChannel.java:50) app//org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:430) app//org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:419) app//org.apache.maven.surefire.api.stream.AbstractStreamDecoder.readMessageType(AbstractStreamDecoder.java:116) app//org.apache.maven.surefire.booter.stream.CommandDecoder.decode(CommandDecoder.java:77) app//org.apache.maven.surefire.booter.spi.CommandChannelDecoder.decode(CommandChannelDecoder.java:60) app//org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:290) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 20 (Listener at localhost/35085): State: TIMED_WAITING Blocked count: 208 Waited count: 1358 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1798) java.base@11.0.17/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128) java.base@11.0.17/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1868) java.base@11.0.17/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2021) app//org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:196) app//org.apache.hadoop.hbase.client.Admin.removeReplicationPeer(Admin.java:2007) app//org.apache.hadoop.hbase.replication.SyncReplicationTestBase.shutdown(SyncReplicationTestBase.java:152) app//org.apache.hadoop.hbase.replication.SyncReplicationTestBase.tearDown(SyncReplicationTestBase.java:160) java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) java.base@11.0.17/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.base@11.0.17/java.lang.reflect.Method.invoke(Method.java:566) app//org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) app//org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) app//org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) app//org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) app//org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) app//org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) Thread 22 (NIOServerCxnFactory.SelectorThread-0): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.select(NIOServerCnxnFactory.java:426) app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.run(NIOServerCnxnFactory.java:391) Thread 23 (NIOServerCxnFactory.SelectorThread-1): State: RUNNABLE Blocked count: 15 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.select(NIOServerCnxnFactory.java:426) app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.run(NIOServerCnxnFactory.java:391) Thread 24 (NIOServerCxnFactory.AcceptThread:localhost/127.0.0.1:52753): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.zookeeper.server.NIOServerCnxnFactory$AcceptThread.select(NIOServerCnxnFactory.java:229) app//org.apache.zookeeper.server.NIOServerCnxnFactory$AcceptThread.run(NIOServerCnxnFactory.java:205) Thread 21 (ConnnectionExpirer): State: TIMED_WAITING Blocked count: 0 Waited count: 124 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.zookeeper.server.NIOServerCnxnFactory$ConnectionExpirerThread.run(NIOServerCnxnFactory.java:574) Thread 25 (SessionTracker): State: TIMED_WAITING Blocked count: 0 Waited count: 619 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Thread 26 (SyncThread:0): State: WAITING Blocked count: 10 Waited count: 1106 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4783b3b2 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:109) Thread 27 (ProcessThread(sid:0 cport:52753):): State: WAITING Blocked count: 2 Waited count: 1174 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@619c75b6 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:134) Thread 28 (NIOWorkerThread-1): State: WAITING Blocked count: 2 Waited count: 171 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 29 (org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner): State: WAITING Blocked count: 4 Waited count: 5 Waiting on java.lang.ref.ReferenceQueue$Lock@59bc590c Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176) app//org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3839) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 122 (IPC Parameter Sending Thread #0): State: TIMED_WAITING Blocked count: 0 Waited count: 2098 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462) java.base@11.0.17/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361) java.base@11.0.17/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 193 (ForkJoinPool-2-worker-13): State: WAITING Blocked count: 0 Waited count: 2 Waiting on java.util.concurrent.ForkJoinPool@489573f3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628) java.base@11.0.17/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) Thread 241 (weak-ref-cleaner-strictcontextstorage): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.lang.ref.ReferenceQueue$Lock@4e8e353b Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176) app//io.opentelemetry.context.StrictContextStorage$PendingScopes.run(StrictContextStorage.java:269) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 242 (HBase-Metrics2-1): State: TIMED_WAITING Blocked count: 0 Waited count: 1541 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 243 (HMaster-EventLoopGroup-1-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 246 (NIOWorkerThread-2): State: WAITING Blocked count: 1 Waited count: 171 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 247 (NIOWorkerThread-3): State: WAITING Blocked count: 3 Waited count: 173 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 248 (NIOWorkerThread-4): State: WAITING Blocked count: 2 Waited count: 171 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 250 (NIOWorkerThread-5): State: WAITING Blocked count: 1 Waited count: 170 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 251 (NIOWorkerThread-6): State: WAITING Blocked count: 1 Waited count: 170 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 252 (NIOWorkerThread-7): State: WAITING Blocked count: 3 Waited count: 170 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 253 (NIOWorkerThread-8): State: WAITING Blocked count: 2 Waited count: 172 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 254 (NIOWorkerThread-9): State: WAITING Blocked count: 3 Waited count: 170 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 255 (NIOWorkerThread-10): State: WAITING Blocked count: 1 Waited count: 172 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 256 (NIOWorkerThread-11): State: WAITING Blocked count: 3 Waited count: 170 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 257 (NIOWorkerThread-12): State: WAITING Blocked count: 1 Waited count: 171 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 258 (NIOWorkerThread-13): State: WAITING Blocked count: 2 Waited count: 172 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 259 (NIOWorkerThread-14): State: WAITING Blocked count: 1 Waited count: 171 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 260 (NIOWorkerThread-15): State: WAITING Blocked count: 0 Waited count: 171 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 261 (NIOWorkerThread-16): State: WAITING Blocked count: 3 Waited count: 169 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 275 (HMaster-EventLoopGroup-3-1): State: RUNNABLE Blocked count: 12 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 292 (Listener at localhost/38557.named-queue-events-pool-0): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@27b648e5 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) app//com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:47) app//com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56) app//com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:159) app//com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 293 (MiniHBaseClusterRegionServer-EventLoopGroup-4-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 315 (MiniHBaseClusterRegionServer-EventLoopGroup-5-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 337 (MiniHBaseClusterRegionServer-EventLoopGroup-6-1): State: RUNNABLE Blocked count: 53 Waited count: 10 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 362 (Monitor thread for TaskMonitor): State: TIMED_WAITING Blocked count: 0 Waited count: 123 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:325) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 364 (master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics): State: TIMED_WAITING Blocked count: 0 Waited count: 5 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 366 (master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics): State: TIMED_WAITING Blocked count: 0 Waited count: 5 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 368 (org.apache.hadoop.hdfs.PeerCache@29ffa1e2): State: TIMED_WAITING Blocked count: 0 Waited count: 409 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) app//org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) app//org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 404 (MiniHBaseClusterRegionServer-EventLoopGroup-6-2): State: RUNNABLE Blocked count: 61 Waited count: 10 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 405 (MiniHBaseClusterRegionServer-EventLoopGroup-6-3): State: RUNNABLE Blocked count: 58 Waited count: 8 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 418 (Idle-Rpc-Conn-Sweeper-pool-0): State: TIMED_WAITING Blocked count: 0 Waited count: 87 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 429 (SnapshotHandlerChoreCleaner): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4704add8 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 417 (RpcClient-timer-pool-0): State: TIMED_WAITING Blocked count: 0 Waited count: 122462 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 438 (HMaster-EventLoopGroup-3-2): State: RUNNABLE Blocked count: 5 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 439 (HMaster-EventLoopGroup-3-3): State: RUNNABLE Blocked count: 16 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 485 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 4 Waited count: 9 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@72ff010 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 486 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 4 Waited count: 9 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@676d0211 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 484 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 5 Waited count: 9 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@173cf4f0 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 531 (region-location-0): State: WAITING Blocked count: 8 Waited count: 16 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9411df7 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 534 (region-location-1): State: WAITING Blocked count: 0 Waited count: 2 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9411df7 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 536 (RPCClient-NioEventLoopGroup-7-1): State: RUNNABLE Blocked count: 3 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 537 (RPCClient-NioEventLoopGroup-7-2): State: RUNNABLE Blocked count: 7 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 538 (HMaster-EventLoopGroup-1-2): State: RUNNABLE Blocked count: 4 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 416 (Async-Client-Retry-Timer-pool-0): State: TIMED_WAITING Blocked count: 0 Waited count: 122250 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 539 (RPCClient-NioEventLoopGroup-7-3): State: RUNNABLE Blocked count: 14 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 546 (org.apache.hadoop.util.JvmPauseMonitor$Monitor@479fa89): State: TIMED_WAITING Blocked count: 0 Waited count: 2445 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 547 (pool-240-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 548 (qtp651210004-548): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 549 (qtp651210004-549): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 550 (qtp651210004-550): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 551 (qtp651210004-551): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 552 (qtp651210004-552-acceptor-0@161485b2-ServerConnector@76c396f{HTTP/1.1, (http/1.1)}{localhost:33811}): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 553 (qtp651210004-553): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 554 (qtp651210004-554): State: TIMED_WAITING Blocked count: 0 Waited count: 24 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 555 (qtp651210004-555): State: TIMED_WAITING Blocked count: 0 Waited count: 22 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 556 (Session-HouseKeeper-4d1a7915-1): State: TIMED_WAITING Blocked count: 0 Waited count: 2 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 562 (FSEditLogAsync): State: WAITING Blocked count: 6 Waited count: 348 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@19d1f8b6 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:417) app//org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.dequeueEdit(FSEditLogAsync.java:235) app//org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.run(FSEditLogAsync.java:244) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 564 (Socket Reader #1 for port 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) Thread 565 (IPC Server idle connection scanner for port 39027): State: TIMED_WAITING Blocked count: 1 Waited count: 124 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 569 (org.apache.hadoop.hdfs.server.blockmanagement.PendingReconstructionBlocks$PendingReconstructionMonitor@1d7dee8b): State: TIMED_WAITING Blocked count: 0 Waited count: 5 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.blockmanagement.PendingReconstructionBlocks$PendingReconstructionMonitor.run(PendingReconstructionBlocks.java:259) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 570 (DatanodeAdminMonitor-0): State: TIMED_WAITING Blocked count: 0 Waited count: 408 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 558 (org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor@6315f161): State: TIMED_WAITING Blocked count: 0 Waited count: 245 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor.run(HeartbeatManager.java:541) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 557 (RedundancyMonitor): State: TIMED_WAITING Blocked count: 0 Waited count: 408 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) java.base@11.0.17/java.lang.Thread.sleep(Thread.java:334) java.base@11.0.17/java.util.concurrent.TimeUnit.sleep(TimeUnit.java:446) app//org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$RedundancyMonitor.run(BlockManager.java:4833) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 560 (Block report processor): State: WAITING Blocked count: 0 Waited count: 168 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7119fcd6 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:417) app//org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.processQueue(BlockManager.java:5107) app//org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.run(BlockManager.java:5096) Thread 566 (IPC Server Responder): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) Thread 563 (IPC Server listener on 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) Thread 572 (IPC Server handler 0 on default port 39027): State: TIMED_WAITING Blocked count: 12 Waited count: 1587 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 573 (IPC Server handler 1 on default port 39027): State: TIMED_WAITING Blocked count: 12 Waited count: 1588 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 574 (IPC Server handler 2 on default port 39027): State: TIMED_WAITING Blocked count: 14 Waited count: 1583 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 575 (IPC Server handler 3 on default port 39027): State: TIMED_WAITING Blocked count: 12 Waited count: 1579 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 576 (IPC Server handler 4 on default port 39027): State: TIMED_WAITING Blocked count: 9 Waited count: 1591 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 577 (pool-242-thread-1): State: TIMED_WAITING Blocked count: 3 Waited count: 4 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 579 (org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor@429a6ebe): State: TIMED_WAITING Blocked count: 0 Waited count: 612 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor.run(LeaseManager.java:526) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 580 (org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor@53f681f5): State: TIMED_WAITING Blocked count: 0 Waited count: 245 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor.run(FSNamesystem.java:4093) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 581 (org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeEditLogRoller@6d64b734): State: TIMED_WAITING Blocked count: 0 Waited count: 5 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeEditLogRoller.run(FSNamesystem.java:4135) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 582 (org.apache.hadoop.hdfs.server.namenode.FSNamesystem$LazyPersistFileScrubber@1482f030): State: TIMED_WAITING Blocked count: 0 Waited count: 5 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.namenode.FSNamesystem$LazyPersistFileScrubber.run(FSNamesystem.java:4228) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 583 (CacheReplicationMonitor(2132964546)): State: TIMED_WAITING Blocked count: 0 Waited count: 41 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2211) app//org.apache.hadoop.hdfs.server.blockmanagement.CacheReplicationMonitor.run(CacheReplicationMonitor.java:181) Thread 592 (pool-248-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 593 (qtp1531759665-593): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 594 (qtp1531759665-594-acceptor-0@7dfb85d8-ServerConnector@7aafb906{HTTP/1.1, (http/1.1)}{localhost:39221}): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 595 (qtp1531759665-595): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 596 (qtp1531759665-596): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 597 (Session-HouseKeeper-4e3199e3-1): State: TIMED_WAITING Blocked count: 0 Waited count: 2 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 598 (nioEventLoopGroup-8-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:803) app//io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457) app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 599 (org.apache.hadoop.util.JvmPauseMonitor$Monitor@3a0ed71): State: TIMED_WAITING Blocked count: 0 Waited count: 2444 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 601 (Socket Reader #1 for port 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) Thread 602 (IPC Server idle connection scanner for port 35753): State: TIMED_WAITING Blocked count: 1 Waited count: 124 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 606 (Command processor): State: WAITING Blocked count: 0 Waited count: 411 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@cdef922 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.processQueue(BPServiceActor.java:1291) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.run(BPServiceActor.java:1275) Thread 607 (BP-1531809096-172.17.0.3-1697385856100 heartbeating to localhost/127.0.0.1:39027): State: TIMED_WAITING Blocked count: 504 Waited count: 1372 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager.waitTillNextIBR(IncrementalBlockReportManager.java:158) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:719) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:855) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 608 (pool-249-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 591 (org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@44ecbf13): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) java.base@11.0.17/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:114) app//org.apache.hadoop.hdfs.net.TcpPeerServer.accept(TcpPeerServer.java:85) app//org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:145) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 603 (IPC Server Responder): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) Thread 609 (IPC Client (306965244) connection to localhost/127.0.0.1:39027 from jenkins): State: TIMED_WAITING Blocked count: 1332 Waited count: 1333 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1086) app//org.apache.hadoop.ipc.Client$Connection.run(Client.java:1133) Thread 600 (IPC Server listener on 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) Thread 610 (IPC Server handler 0 on default port 35753): State: TIMED_WAITING Blocked count: 0 Waited count: 1227 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 611 (IPC Server handler 1 on default port 35753): State: TIMED_WAITING Blocked count: 0 Waited count: 1227 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 612 (IPC Server handler 2 on default port 35753): State: TIMED_WAITING Blocked count: 0 Waited count: 1226 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 613 (IPC Server handler 3 on default port 35753): State: TIMED_WAITING Blocked count: 0 Waited count: 1226 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 614 (IPC Server handler 4 on default port 35753): State: TIMED_WAITING Blocked count: 0 Waited count: 1226 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 623 (pool-256-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 624 (qtp315182252-624): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 625 (qtp315182252-625-acceptor-0@109265bd-ServerConnector@69e73e0b{HTTP/1.1, (http/1.1)}{localhost:45307}): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 626 (qtp315182252-626): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 627 (qtp315182252-627): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 629 (Session-HouseKeeper-5bb3f6b3-1): State: TIMED_WAITING Blocked count: 0 Waited count: 3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 631 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 632 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 637 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 1 Waited count: 3 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 638 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 1 Waited count: 3 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 642 (pool-245-thread-1): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@298da77 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 645 (java.util.concurrent.ThreadPoolExecutor$Worker@652247d8[State = -1, empty queue]): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 646 (nioEventLoopGroup-10-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:803) app//io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457) app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 647 (org.apache.hadoop.util.JvmPauseMonitor$Monitor@6f1e007b): State: TIMED_WAITING Blocked count: 0 Waited count: 2444 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 649 (Socket Reader #1 for port 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) Thread 650 (IPC Server idle connection scanner for port 44717): State: TIMED_WAITING Blocked count: 1 Waited count: 124 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 654 (Command processor): State: WAITING Blocked count: 0 Waited count: 410 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1bf76d01 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.processQueue(BPServiceActor.java:1291) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.run(BPServiceActor.java:1275) Thread 655 (BP-1531809096-172.17.0.3-1697385856100 heartbeating to localhost/127.0.0.1:39027): State: TIMED_WAITING Blocked count: 510 Waited count: 1369 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager.waitTillNextIBR(IncrementalBlockReportManager.java:158) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:719) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:855) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 656 (pool-261-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 622 (org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@57b6801e): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) java.base@11.0.17/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:114) app//org.apache.hadoop.hdfs.net.TcpPeerServer.accept(TcpPeerServer.java:85) app//org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:145) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 651 (IPC Server Responder): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) Thread 648 (IPC Server listener on 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) Thread 657 (IPC Server handler 0 on default port 44717): State: TIMED_WAITING Blocked count: 0 Waited count: 1226 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 658 (IPC Server handler 1 on default port 44717): State: TIMED_WAITING Blocked count: 0 Waited count: 1226 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 659 (IPC Server handler 2 on default port 44717): State: TIMED_WAITING Blocked count: 0 Waited count: 1227 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 660 (IPC Server handler 3 on default port 44717): State: TIMED_WAITING Blocked count: 0 Waited count: 1226 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 661 (IPC Server handler 4 on default port 44717): State: TIMED_WAITING Blocked count: 0 Waited count: 1226 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 670 (pool-268-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 671 (qtp1875028383-671): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 672 (qtp1875028383-672-acceptor-0@49fdfdbd-ServerConnector@692e91fb{HTTP/1.1, (http/1.1)}{localhost:35369}): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 673 (qtp1875028383-673): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 674 (qtp1875028383-674): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 675 (Session-HouseKeeper-19accb11-1): State: TIMED_WAITING Blocked count: 0 Waited count: 2 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 678 (nioEventLoopGroup-12-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:803) app//io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457) app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 679 (org.apache.hadoop.util.JvmPauseMonitor$Monitor@4fd4121a): State: TIMED_WAITING Blocked count: 0 Waited count: 2443 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 681 (Socket Reader #1 for port 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) Thread 682 (IPC Server idle connection scanner for port 35085): State: TIMED_WAITING Blocked count: 1 Waited count: 124 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 686 (Command processor): State: WAITING Blocked count: 0 Waited count: 409 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@50974f58 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.processQueue(BPServiceActor.java:1291) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.run(BPServiceActor.java:1275) Thread 688 (BP-1531809096-172.17.0.3-1697385856100 heartbeating to localhost/127.0.0.1:39027): State: TIMED_WAITING Blocked count: 503 Waited count: 1358 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager.waitTillNextIBR(IncrementalBlockReportManager.java:158) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:719) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:855) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 687 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 689 (pool-270-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 669 (org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@379268ae): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) java.base@11.0.17/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:114) app//org.apache.hadoop.hdfs.net.TcpPeerServer.accept(TcpPeerServer.java:85) app//org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:145) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 683 (IPC Server Responder): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) Thread 690 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 680 (IPC Server listener on 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) Thread 693 (IPC Server handler 0 on default port 35085): State: TIMED_WAITING Blocked count: 0 Waited count: 1226 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 696 (IPC Server handler 1 on default port 35085): State: TIMED_WAITING Blocked count: 0 Waited count: 1226 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 695 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 1 Waited count: 3 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 697 (IPC Server handler 2 on default port 35085): State: TIMED_WAITING Blocked count: 0 Waited count: 1226 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 698 (IPC Server handler 3 on default port 35085): State: TIMED_WAITING Blocked count: 0 Waited count: 1225 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 699 (IPC Server handler 4 on default port 35085): State: TIMED_WAITING Blocked count: 0 Waited count: 1225 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 702 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 1 Waited count: 3 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 706 (pool-253-thread-1): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@46744811 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 710 (java.util.concurrent.ThreadPoolExecutor$Worker@5054e7fc[State = -1, empty queue]): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 713 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 714 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 719 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 2 Waited count: 5 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 720 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 3 Waited count: 5 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 724 (pool-265-thread-1): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@44ff5530 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 728 (java.util.concurrent.ThreadPoolExecutor$Worker@5593a5f4[State = -1, empty queue]): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 729 (FsDatasetAsyncDiskServiceFixer): State: TIMED_WAITING Blocked count: 0 Waited count: 41 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer.run(HBaseTestingUtil.java:596) Thread 731 (LeaseRenewer:jenkins@localhost:39027): State: TIMED_WAITING Blocked count: 40 Waited count: 1301 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:441) app//org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$800(LeaseRenewer.java:77) app//org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:336) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 740 (HMaster-EventLoopGroup-8-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 741 (Listener at localhost/35085-SendThread(127.0.0.1:52753)): State: RUNNABLE Blocked count: 8 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Thread 742 (Listener at localhost/35085-EventThread): State: WAITING Blocked count: 7 Waited count: 22 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1053737 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Thread 743 (zk-event-processor-pool-0): State: WAITING Blocked count: 21 Waited count: 39 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@488db1fb Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 745 (RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@76652b0f Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 746 (RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.Semaphore$NonfairSync@5121d015 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 747 (RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213): State: WAITING Blocked count: 4 Waited count: 12 Waiting on java.util.concurrent.Semaphore$NonfairSync@7935e989 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 748 (RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c929d3d Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 749 (RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3f0f8f3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 750 (RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3f0f8f3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 751 (RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@5c0e21e0 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 752 (RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@a21f249 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 753 (RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@4292fd45 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 754 (RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@4621b284 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 757 (HMaster-EventLoopGroup-9-1): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 758 (Listener at localhost/35085-SendThread(127.0.0.1:52753)): State: RUNNABLE Blocked count: 10 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Thread 759 (Listener at localhost/35085-EventThread): State: WAITING Blocked count: 4 Waited count: 23 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6b0d9a81 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Thread 760 (zk-event-processor-pool-0): State: WAITING Blocked count: 19 Waited count: 40 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ebb9e13 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 762 (RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403): State: WAITING Blocked count: 7 Waited count: 68 Waiting on java.util.concurrent.Semaphore$NonfairSync@3b963e54 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 763 (RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403): State: WAITING Blocked count: 15 Waited count: 56 Waiting on java.util.concurrent.Semaphore$NonfairSync@70681e30 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 764 (RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403): State: WAITING Blocked count: 1 Waited count: 35 Waiting on java.util.concurrent.Semaphore$NonfairSync@77b65b2e Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 765 (RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33403): State: WAITING Blocked count: 1 Waited count: 629 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@454f18cd Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 766 (RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33403): State: WAITING Blocked count: 6 Waited count: 4 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4fa1faae Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 767 (RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33403): State: WAITING Blocked count: 0 Waited count: 4 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4fa1faae Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 768 (RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=33403): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@7f08f4f4 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 769 (RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=33403): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@3aa2bf10 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 770 (RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33403): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@79651f9 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 771 (RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=33403): State: WAITING Blocked count: 0 Waited count: 7 Waiting on java.util.concurrent.Semaphore$NonfairSync@47a2af2f Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 774 (MiniHBaseClusterRegionServer-EventLoopGroup-10-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 796 (MiniHBaseClusterRegionServer-EventLoopGroup-11-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 818 (MiniHBaseClusterRegionServer-EventLoopGroup-12-1): State: RUNNABLE Blocked count: 9 Waited count: 5 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:312) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:376) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 755 (M:0;8c5d72df62de:38213): State: TIMED_WAITING Blocked count: 9 Waited count: 12184 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) app//org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:623) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 772 (M:1;8c5d72df62de:33403): State: TIMED_WAITING Blocked count: 17 Waited count: 12186 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) app//org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:623) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 841 (master/8c5d72df62de:0:becomeActiveMaster): State: TIMED_WAITING Blocked count: 5 Waited count: 19 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hbase.master.ActiveMasterManager.blockUntilBecomingActiveMaster(ActiveMasterManager.java:305) app//org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2486) app//org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:611) app//org.apache.hadoop.hbase.master.HMaster$$Lambda$291/0x0000000800958c40.run(Unknown Source) app//org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) app//org.apache.hadoop.hbase.trace.TraceUtil$$Lambda$276/0x000000080075d440.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 861 (master:store-WAL-Roller): State: TIMED_WAITING Blocked count: 4 Waited count: 12187 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hbase.wal.AbstractWALRoller.run(AbstractWALRoller.java:180) Thread 878 (MiniHBaseClusterRegionServer-EventLoopGroup-12-2): State: RUNNABLE Blocked count: 10 Waited count: 7 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:312) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:376) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 879 (MiniHBaseClusterRegionServer-EventLoopGroup-12-3): State: RUNNABLE Blocked count: 7 Waited count: 4 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:312) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:376) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 886 (AsyncFSWAL-0-hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData-prefix:8c5d72df62de,33403,1697385859093): State: WAITING Blocked count: 0 Waited count: 1219 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@49839aec Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 889 (master:store-Flusher): State: TIMED_WAITING Blocked count: 62 Waited count: 108 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2211) app//org.apache.hadoop.hbase.master.region.MasterRegionFlusherAndCompactor.flushLoop(MasterRegionFlusherAndCompactor.java:193) app//org.apache.hadoop.hbase.master.region.MasterRegionFlusherAndCompactor$$Lambda$383/0x0000000800a05840.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 897 (ProcedureDispatcherTimeoutThread): State: TIMED_WAITING Blocked count: 0 Waited count: 111 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) app//org.apache.hadoop.hbase.procedure2.util.DelayedUtil.takeWithoutInterrupt(DelayedUtil.java:81) app//org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher$TimeoutExecutorThread.run(RemoteProcedureDispatcher.java:320) Thread 898 (8c5d72df62de:33403): State: WAITING Blocked count: 4 Waited count: 25 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a323a27 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) app//org.apache.hadoop.hbase.master.assignment.AssignmentManager.waitOnAssignQueue(AssignmentManager.java:2242) app//org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:2264) app//org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$600(AssignmentManager.java:111) app//org.apache.hadoop.hbase.master.assignment.AssignmentManager$1.run(AssignmentManager.java:2204) Thread 899 (normalizer-worker-0): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3fb38977 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) app//org.apache.hadoop.hbase.master.normalizer.RegionNormalizerWorkQueue.take(RegionNormalizerWorkQueue.java:146) app//org.apache.hadoop.hbase.master.normalizer.RegionNormalizerWorker.run(RegionNormalizerWorker.java:191) java.base@11.0.17/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) java.base@11.0.17/java.util.concurrent.FutureTask.run(FutureTask.java:264) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 900 (SnapshotHandlerChoreCleaner): State: TIMED_WAITING Blocked count: 0 Waited count: 122 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 890 (ProcExecTimeout): State: TIMED_WAITING Blocked count: 1 Waited count: 184 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:274) app//org.apache.hadoop.hbase.procedure2.util.DelayedUtil.takeWithoutInterrupt(DelayedUtil.java:81) app//org.apache.hadoop.hbase.procedure2.TimeoutExecutorThread.run(TimeoutExecutorThread.java:56) Thread 891 (WorkerMonitor): State: TIMED_WAITING Blocked count: 0 Waited count: 245 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:279) app//org.apache.hadoop.hbase.procedure2.util.DelayedUtil.takeWithoutInterrupt(DelayedUtil.java:81) app//org.apache.hadoop.hbase.procedure2.TimeoutExecutorThread.run(TimeoutExecutorThread.java:56) Thread 892 (PEWorker-1): State: TIMED_WAITING Blocked count: 84 Waited count: 310 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) Thread 893 (PEWorker-2): State: TIMED_WAITING Blocked count: 48 Waited count: 257 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) Thread 894 (PEWorker-3): State: TIMED_WAITING Blocked count: 54 Waited count: 237 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) Thread 895 (PEWorker-4): State: TIMED_WAITING Blocked count: 72 Waited count: 234 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) Thread 896 (PEWorker-5): State: TIMED_WAITING Blocked count: 33 Waited count: 189 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) Thread 903 (OldWALsCleaner-0): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4e9b90b3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.master.cleaner.LogCleaner.deleteFile(LogCleaner.java:172) app//org.apache.hadoop.hbase.master.cleaner.LogCleaner.lambda$createOldWalsCleaner$1(LogCleaner.java:152) app//org.apache.hadoop.hbase.master.cleaner.LogCleaner$$Lambda$511/0x0000000800ad5040.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 904 (master/8c5d72df62de:0.Chore.1): State: WAITING Blocked count: 33 Waited count: 90 Waiting on org.apache.hadoop.hbase.client.AsyncTableResultScanner@21d0118e Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.Object.wait(Object.java:328) app//org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:156) app//org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:493) app//org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:461) app//org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:422) app//org.apache.hadoop.hbase.MetaTableAccessor.fullScanTables(MetaTableAccessor.java:126) app//org.apache.hadoop.hbase.master.TableStateManager.getTablesInStates(TableStateManager.java:121) app//org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:2194) app//org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:48) app//org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) app//org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) app//org.apache.hadoop.hbase.trace.TraceUtil$$Lambda$276/0x000000080075d440.run(Unknown Source) java.base@11.0.17/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) java.base@11.0.17/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) app//org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 905 (snapshot-hfile-cleaner-cache-refresher): State: TIMED_WAITING Blocked count: 4 Waited count: 13 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 906 (master/8c5d72df62de:0:becomeActiveMaster-HFileCleaner.large.0-1697385860384): State: WAITING Blocked count: 2 Waited count: 5 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3e23859b Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) app//org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:285) app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:254) Thread 907 (master/8c5d72df62de:0:becomeActiveMaster-HFileCleaner.small.0-1697385860393): State: WAITING Blocked count: 2 Waited count: 6 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2caf94e4 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:546) app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:285) app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner$2.run(HFileCleaner.java:269) Thread 922 (HMaster-EventLoopGroup-9-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 923 (HMaster-EventLoopGroup-9-3): State: RUNNABLE Blocked count: 3 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 928 (RegionServerTracker-0): State: WAITING Blocked count: 6 Waited count: 12 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5cb489a8 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 927 (RegionServerTracker-0): State: WAITING Blocked count: 11 Waited count: 24 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@765f54f4 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 954 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 8 Waited count: 17 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2b051542 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 949 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 11 Waited count: 23 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@768cf997 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 957 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 4 Waited count: 9 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2fb61145 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 1005 (region-location-0): State: WAITING Blocked count: 3 Waited count: 6 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3de26df3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1006 (Registry-endpoints-refresh-end-points): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hbase.client.RegistryEndpointsRefresher.mainLoop(RegistryEndpointsRefresher.java:92) app//org.apache.hadoop.hbase.client.RegistryEndpointsRefresher$$Lambda$698/0x0000000800bb0840.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1007 (RPCClient-NioEventLoopGroup-7-4): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1008 (RPCClient-NioEventLoopGroup-7-5): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1009 (HMaster-EventLoopGroup-8-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1010 (RPCClient-NioEventLoopGroup-7-6): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1011 (RPCClient-NioEventLoopGroup-7-7): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1034 (RPCClient-NioEventLoopGroup-7-8): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1078 (MiniHBaseClusterRegionServer-EventLoopGroup-4-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1090 (MiniHBaseClusterRegionServer-EventLoopGroup-4-3): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1093 (MiniHBaseClusterRegionServer-EventLoopGroup-5-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1159 (MiniHBaseClusterRegionServer-EventLoopGroup-10-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1175 (MiniHBaseClusterRegionServer-EventLoopGroup-10-3): State: RUNNABLE Blocked count: 3 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1179 (MiniHBaseClusterRegionServer-EventLoopGroup-11-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1644 (RegionServer-Restarter): State: TIMED_WAITING Blocked count: 9 Waited count: 1215 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hbase.replication.TestSyncReplicationStandbyKillRS.waitForRSShutdownToStartAndFinish(TestSyncReplicationStandbyKillRS.java:137) app//org.apache.hadoop.hbase.replication.TestSyncReplicationStandbyKillRS.lambda$testStandbyKillRegionServer$1(TestSyncReplicationStandbyKillRS.java:82) app//org.apache.hadoop.hbase.replication.TestSyncReplicationStandbyKillRS$$Lambda$960/0x0000000800ca7040.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1704 (HMaster-EventLoopGroup-8-3): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1738 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175-SendThread(127.0.0.1:52753)): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Thread 1739 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175-EventThread): State: WAITING Blocked count: 0 Waited count: 2 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7460149d Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Thread 1740 (ReadOnlyZKClient-127.0.0.1:52753@0x365ad98b): State: TIMED_WAITING Blocked count: 2 Waited count: 23 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:328) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$813/0x0000000800c14040.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1743 (zk-event-processor-pool-0): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@615393cd Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1747 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175): State: WAITING Blocked count: 5 Waited count: 9 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@15e3b786 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:341) app//org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.replicationDone(ReplicationSourceWALReader.java:121) app//org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:149) Thread 1749 (Abort regionserver monitor): State: RUNNABLE Blocked count: 1 Waited count: 2 Stack: java.management@11.0.17/sun.management.ThreadImpl.getThreadInfo1(Native Method) java.management@11.0.17/sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:197) java.management@11.0.17/sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:154) app//org.apache.hadoop.hbase.util.ReflectionUtils.printThreadInfo(ReflectionUtils.java:175) app//org.apache.hadoop.hbase.util.Threads.printThreadInfo(Threads.java:186) app//org.apache.hadoop.hbase.regionserver.HRegionServer$SystemExitWhenAbortTimeout.run(HRegionServer.java:3564) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:556) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 1871 (MiniHBaseClusterRegionServer-EventLoopGroup-11-3): State: RUNNABLE Blocked count: 4 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1953 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175-SendThread(127.0.0.1:52753)): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Thread 1954 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175-EventThread): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2045228c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Thread 1955 (ReadOnlyZKClient-127.0.0.1:52753@0x0072515b): State: TIMED_WAITING Blocked count: 2 Waited count: 23 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:328) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$813/0x0000000800c14040.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1956 (zk-event-processor-pool-0): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@53b14619 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1966 (MiniHBaseClusterRegionServer-EventLoopGroup-13-1): State: RUNNABLE Blocked count: 3 Waited count: 2 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1989 (MiniHBaseClusterRegionServer-EventLoopGroup-13-2): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1997 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 1 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@99f54bd Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 2041 (MiniHBaseClusterRegionServer-EventLoopGroup-13-3): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2156 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222-SendThread(127.0.0.1:52753)): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Thread 2157 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222-EventThread): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@336604e0 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Thread 2158 (ReadOnlyZKClient-127.0.0.1:52753@0x0a7feabd): State: TIMED_WAITING Blocked count: 2 Waited count: 24 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:328) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$813/0x0000000800c14040.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2159 (zk-event-processor-pool-0): State: WAITING Blocked count: 0 Waited count: 2 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2cdd34ce Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2166 (Abort regionserver monitor): State: TIMED_WAITING Blocked count: 1 Waited count: 1 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 2216 (master/8c5d72df62de:0.Chore.2): State: WAITING Blocked count: 2 Waited count: 5 Waiting on java.util.concurrent.CompletableFuture$Signaller@61561955 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1796) java.base@11.0.17/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128) java.base@11.0.17/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1823) java.base@11.0.17/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1998) app//org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:182) app//org.apache.hadoop.hbase.client.AdminOverAsyncAdmin.tableExists(AdminOverAsyncAdmin.java:130) app//org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.hasData(TableReplicationQueueStorage.java:536) app//org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.preClean(ReplicationLogCleaner.java:81) app//org.apache.hadoop.hbase.master.cleaner.CleanerChore$$Lambda$1116/0x0000000800d55840.accept(Unknown Source) java.base@11.0.17/java.util.ArrayList.forEach(ArrayList.java:1541) app//org.apache.hadoop.hbase.master.cleaner.CleanerChore.preRunCleaner(CleanerChore.java:282) app//org.apache.hadoop.hbase.master.cleaner.CleanerChore.chore(CleanerChore.java:257) app//org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) app//org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) app//org.apache.hadoop.hbase.trace.TraceUtil$$Lambda$276/0x000000080075d440.run(Unknown Source) java.base@11.0.17/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) java.base@11.0.17/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) Thread 2247 (RPCClient-NioEventLoopGroup-7-9): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2248 (Time-limited test): State: TIMED_WAITING Blocked count: 6 Waited count: 3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1798) java.base@11.0.17/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128) java.base@11.0.17/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1868) java.base@11.0.17/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2021) app//org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:196) app//org.apache.hadoop.hbase.client.Admin.removeReplicationPeer(Admin.java:2007) app//org.apache.hadoop.hbase.replication.SyncReplicationTestBase.shutdown(SyncReplicationTestBase.java:152) app//org.apache.hadoop.hbase.replication.SyncReplicationTestBase.tearDown(SyncReplicationTestBase.java:160) java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) java.base@11.0.17/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.base@11.0.17/java.lang.reflect.Method.invoke(Method.java:566) app//org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) app//org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) app//org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) app//org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) app//org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) app//org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) Thread 2260 (DataXceiver for client DFSClient_NONMAPREDUCE_-1007166148_20 at /127.0.0.1:44644 [Receiving block BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051]): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) app//org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:156) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) java.base@11.0.17/java.io.DataInputStream.read(DataInputStream.java:149) app//org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:210) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:971) app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:908) app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:173) Thread 2262 (DataXceiver for client DFSClient_NONMAPREDUCE_-1007166148_20 at /127.0.0.1:47764 [Receiving block BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051]): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) app//org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:156) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) java.base@11.0.17/java.io.DataInputStream.read(DataInputStream.java:149) app//org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:210) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:971) app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:908) app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:173) Thread 2261 (DataXceiver for client DFSClient_NONMAPREDUCE_-1007166148_20 at /127.0.0.1:53036 [Receiving block BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051]): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) app//org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:156) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) java.base@11.0.17/java.io.DataInputStream.read(DataInputStream.java:149) app//org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:210) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:971) app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:908) app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:173) Thread 2269 (PacketResponder: BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051, type=LAST_IN_PIPELINE): State: WAITING Blocked count: 14 Waited count: 15 Waiting on java.util.LinkedList@215eb79c Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.Object.wait(Object.java:328) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1330) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1402) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2268 (PacketResponder: BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051, type=LAST_IN_PIPELINE): State: WAITING Blocked count: 14 Waited count: 15 Waiting on java.util.LinkedList@70139e53 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.Object.wait(Object.java:328) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1330) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1402) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2267 (PacketResponder: BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051, type=LAST_IN_PIPELINE): State: WAITING Blocked count: 14 Waited count: 15 Waiting on java.util.LinkedList@17ee7c41 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.Object.wait(Object.java:328) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1330) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1402) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2303 (process reaper): State: TIMED_WAITING Blocked count: 0 Waited count: 2 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462) java.base@11.0.17/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361) java.base@11.0.17/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2305 (Timer for 'DataNode' metrics system): State: TIMED_WAITING Blocked count: 0 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) 2023-10-15T16:24:39,120 ERROR [Abort regionserver monitor {}] server.NIOServerCnxnFactory$1(92): Thread Thread[Abort regionserver monitor,5,FailOnTimeoutGroup] died org.apache.hadoop.hbase.SystemExitRule$SystemExitInTestException: null at org.apache.hadoop.hbase.TestSecurityManager.checkExit(TestSecurityManager.java:32) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT-tests.jar:4.0.0-alpha-1-SNAPSHOT] at java.lang.Runtime.halt(Runtime.java:277) ~[?:?] at org.apache.hadoop.hbase.regionserver.HRegionServer$SystemExitWhenAbortTimeout.run(HRegionServer.java:3565) ~[classes/:?] at java.util.TimerThread.mainLoop(Timer.java:556) ~[?:?] at java.util.TimerThread.run(Timer.java:506) ~[?:?] 2023-10-15T16:24:39,313 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:40,269 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:40,270 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:40,270 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:24:40,271 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18281 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:24:40,271 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:40,271 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:24:40,271 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:40,271 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:40,313 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:41,313 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:42,314 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:43,314 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:44,314 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:45,314 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:46,315 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:47,315 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:48,315 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:48,760 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:24:49,315 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:50,310 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:50,310 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:50,311 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:24:50,311 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28322 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:24:50,312 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:50,312 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:24:50,312 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:50,312 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:50,315 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:51,315 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:52,316 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:53,316 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:54,316 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:54,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:24:54,661 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:24:54,677 WARN [Abort regionserver monitor {}] regionserver.HRegionServer$SystemExitWhenAbortTimeout(3561): Aborting region server timed out, terminating forcibly and does not wait for any running shutdown hooks or finalizers to finish their work. Thread dump to stdout. Process Thread Dump: Zombie HRegionServer 290 active threads Thread 1 (main): State: TIMED_WAITING Blocked count: 7 Waited count: 5 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.FutureTask.awaitDone(FutureTask.java:444) java.base@11.0.17/java.util.concurrent.FutureTask.get(FutureTask.java:203) app//org.junit.internal.runners.statements.FailOnTimeout.getResult(FailOnTimeout.java:167) app//org.junit.internal.runners.statements.FailOnTimeout.evaluate(FailOnTimeout.java:128) app//org.apache.hadoop.hbase.SystemExitRule$1.evaluate(SystemExitRule.java:39) app//org.junit.rules.RunRules.evaluate(RunRules.java:20) app//org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) app//org.junit.runners.ParentRunner.run(ParentRunner.java:413) app//org.junit.runners.Suite.runChild(Suite.java:128) app//org.junit.runners.Suite.runChild(Suite.java:27) app//org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) app//org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) app//org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) app//org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) app//org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) app//org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) app//org.junit.runners.ParentRunner.run(ParentRunner.java:413) app//org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:49) Thread 2 (Reference Handler): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: java.base@11.0.17/java.lang.ref.Reference.waitForReferencePendingList(Native Method) java.base@11.0.17/java.lang.ref.Reference.processPendingReferences(Reference.java:241) java.base@11.0.17/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213) Thread 3 (Finalizer): State: WAITING Blocked count: 12 Waited count: 13 Waiting on java.lang.ref.ReferenceQueue$Lock@1edc134a Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176) java.base@11.0.17/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170) Thread 4 (Signal Dispatcher): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: Thread 11 (Common-Cleaner): State: TIMED_WAITING Blocked count: 12 Waited count: 32 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) java.base@11.0.17/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) java.base@11.0.17/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:161) Thread 12 (pool-1-thread-1): State: RUNNABLE Blocked count: 0 Waited count: 63 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollPort$EventHandlerTask.poll(EPollPort.java:200) java.base@11.0.17/sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:281) java.base@11.0.17/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 13 (pool-1-thread-2): State: WAITING Blocked count: 0 Waited count: 67 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@51ef43d2 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:417) java.base@11.0.17/sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:275) java.base@11.0.17/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 14 (surefire-forkedjvm-stream-flusher): State: TIMED_WAITING Blocked count: 0 Waited count: 12505 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 16 (surefire-forkedjvm-command-thread): State: WAITING Blocked count: 0 Waited count: 128 Waiting on java.util.concurrent.CountDownLatch$Sync@1363f90a Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232) java.base@11.0.17/sun.nio.ch.PendingFuture.get(PendingFuture.java:178) app//org.apache.maven.surefire.api.util.internal.Channels$2.read(Channels.java:127) java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) app//org.apache.maven.surefire.api.util.internal.Channels$3.readImpl(Channels.java:169) app//org.apache.maven.surefire.api.util.internal.AbstractNoninterruptibleReadableChannel.read(AbstractNoninterruptibleReadableChannel.java:50) app//org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:430) app//org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:419) app//org.apache.maven.surefire.api.stream.AbstractStreamDecoder.readMessageType(AbstractStreamDecoder.java:116) app//org.apache.maven.surefire.booter.stream.CommandDecoder.decode(CommandDecoder.java:77) app//org.apache.maven.surefire.booter.spi.CommandChannelDecoder.decode(CommandChannelDecoder.java:60) app//org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:290) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 20 (Listener at localhost/35085): State: TIMED_WAITING Blocked count: 208 Waited count: 1358 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1798) java.base@11.0.17/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128) java.base@11.0.17/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1868) java.base@11.0.17/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2021) app//org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:196) app//org.apache.hadoop.hbase.client.Admin.removeReplicationPeer(Admin.java:2007) app//org.apache.hadoop.hbase.replication.SyncReplicationTestBase.shutdown(SyncReplicationTestBase.java:152) app//org.apache.hadoop.hbase.replication.SyncReplicationTestBase.tearDown(SyncReplicationTestBase.java:160) java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) java.base@11.0.17/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.base@11.0.17/java.lang.reflect.Method.invoke(Method.java:566) app//org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) app//org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) app//org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) app//org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) app//org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) app//org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) Thread 22 (NIOServerCxnFactory.SelectorThread-0): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.select(NIOServerCnxnFactory.java:426) app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.run(NIOServerCnxnFactory.java:391) Thread 23 (NIOServerCxnFactory.SelectorThread-1): State: RUNNABLE Blocked count: 15 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.select(NIOServerCnxnFactory.java:426) app//org.apache.zookeeper.server.NIOServerCnxnFactory$SelectorThread.run(NIOServerCnxnFactory.java:391) Thread 24 (NIOServerCxnFactory.AcceptThread:localhost/127.0.0.1:52753): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.zookeeper.server.NIOServerCnxnFactory$AcceptThread.select(NIOServerCnxnFactory.java:229) app//org.apache.zookeeper.server.NIOServerCnxnFactory$AcceptThread.run(NIOServerCnxnFactory.java:205) Thread 21 (ConnnectionExpirer): State: TIMED_WAITING Blocked count: 0 Waited count: 126 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.zookeeper.server.NIOServerCnxnFactory$ConnectionExpirerThread.run(NIOServerCnxnFactory.java:574) Thread 25 (SessionTracker): State: TIMED_WAITING Blocked count: 0 Waited count: 627 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:151) Thread 26 (SyncThread:0): State: WAITING Blocked count: 10 Waited count: 1112 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4783b3b2 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:109) Thread 27 (ProcessThread(sid:0 cport:52753):): State: WAITING Blocked count: 2 Waited count: 1180 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@619c75b6 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:134) Thread 28 (NIOWorkerThread-1): State: WAITING Blocked count: 2 Waited count: 171 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 29 (org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner): State: WAITING Blocked count: 4 Waited count: 5 Waiting on java.lang.ref.ReferenceQueue$Lock@59bc590c Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176) app//org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3839) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 122 (IPC Parameter Sending Thread #0): State: TIMED_WAITING Blocked count: 0 Waited count: 2114 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462) java.base@11.0.17/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361) java.base@11.0.17/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 193 (ForkJoinPool-2-worker-13): State: WAITING Blocked count: 0 Waited count: 2 Waiting on java.util.concurrent.ForkJoinPool@489573f3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628) java.base@11.0.17/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) Thread 241 (weak-ref-cleaner-strictcontextstorage): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.lang.ref.ReferenceQueue$Lock@4e8e353b Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155) java.base@11.0.17/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176) app//io.opentelemetry.context.StrictContextStorage$PendingScopes.run(StrictContextStorage.java:269) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 242 (HBase-Metrics2-1): State: TIMED_WAITING Blocked count: 0 Waited count: 1558 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 243 (HMaster-EventLoopGroup-1-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 246 (NIOWorkerThread-2): State: WAITING Blocked count: 1 Waited count: 172 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 247 (NIOWorkerThread-3): State: WAITING Blocked count: 3 Waited count: 173 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 248 (NIOWorkerThread-4): State: WAITING Blocked count: 2 Waited count: 172 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 250 (NIOWorkerThread-5): State: WAITING Blocked count: 1 Waited count: 171 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 251 (NIOWorkerThread-6): State: WAITING Blocked count: 1 Waited count: 171 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 252 (NIOWorkerThread-7): State: WAITING Blocked count: 3 Waited count: 170 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 253 (NIOWorkerThread-8): State: WAITING Blocked count: 2 Waited count: 172 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 254 (NIOWorkerThread-9): State: WAITING Blocked count: 3 Waited count: 171 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 255 (NIOWorkerThread-10): State: WAITING Blocked count: 1 Waited count: 173 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 256 (NIOWorkerThread-11): State: WAITING Blocked count: 3 Waited count: 171 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 257 (NIOWorkerThread-12): State: WAITING Blocked count: 1 Waited count: 172 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 258 (NIOWorkerThread-13): State: WAITING Blocked count: 2 Waited count: 173 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 259 (NIOWorkerThread-14): State: WAITING Blocked count: 1 Waited count: 172 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 260 (NIOWorkerThread-15): State: WAITING Blocked count: 0 Waited count: 172 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 261 (NIOWorkerThread-16): State: WAITING Blocked count: 3 Waited count: 170 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@79a4224c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 275 (HMaster-EventLoopGroup-3-1): State: RUNNABLE Blocked count: 12 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 292 (Listener at localhost/38557.named-queue-events-pool-0): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@27b648e5 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) app//com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:47) app//com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56) app//com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:159) app//com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 293 (MiniHBaseClusterRegionServer-EventLoopGroup-4-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 315 (MiniHBaseClusterRegionServer-EventLoopGroup-5-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 337 (MiniHBaseClusterRegionServer-EventLoopGroup-6-1): State: RUNNABLE Blocked count: 53 Waited count: 10 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 362 (Monitor thread for TaskMonitor): State: TIMED_WAITING Blocked count: 0 Waited count: 125 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:325) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 364 (master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics): State: TIMED_WAITING Blocked count: 0 Waited count: 5 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 366 (master/8c5d72df62de:0:becomeActiveMaster-MemStoreChunkPool Statistics): State: TIMED_WAITING Blocked count: 0 Waited count: 5 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 368 (org.apache.hadoop.hdfs.PeerCache@29ffa1e2): State: TIMED_WAITING Blocked count: 0 Waited count: 415 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:253) app//org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:46) app//org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:124) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 404 (MiniHBaseClusterRegionServer-EventLoopGroup-6-2): State: RUNNABLE Blocked count: 61 Waited count: 10 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 405 (MiniHBaseClusterRegionServer-EventLoopGroup-6-3): State: RUNNABLE Blocked count: 58 Waited count: 8 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 418 (Idle-Rpc-Conn-Sweeper-pool-0): State: TIMED_WAITING Blocked count: 0 Waited count: 89 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 429 (SnapshotHandlerChoreCleaner): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4704add8 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 417 (RpcClient-timer-pool-0): State: TIMED_WAITING Blocked count: 0 Waited count: 124026 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 438 (HMaster-EventLoopGroup-3-2): State: RUNNABLE Blocked count: 5 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 439 (HMaster-EventLoopGroup-3-3): State: RUNNABLE Blocked count: 16 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 485 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 4 Waited count: 9 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@72ff010 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 486 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 4 Waited count: 9 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@676d0211 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 484 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 5 Waited count: 9 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@173cf4f0 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 531 (region-location-0): State: WAITING Blocked count: 8 Waited count: 16 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9411df7 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 534 (region-location-1): State: WAITING Blocked count: 0 Waited count: 2 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@9411df7 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 536 (RPCClient-NioEventLoopGroup-7-1): State: RUNNABLE Blocked count: 3 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 537 (RPCClient-NioEventLoopGroup-7-2): State: RUNNABLE Blocked count: 7 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 538 (HMaster-EventLoopGroup-1-2): State: RUNNABLE Blocked count: 4 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 416 (Async-Client-Retry-Timer-pool-0): State: TIMED_WAITING Blocked count: 0 Waited count: 123812 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:600) app//org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:496) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 539 (RPCClient-NioEventLoopGroup-7-3): State: RUNNABLE Blocked count: 14 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 546 (org.apache.hadoop.util.JvmPauseMonitor$Monitor@479fa89): State: TIMED_WAITING Blocked count: 0 Waited count: 2477 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 547 (pool-240-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 548 (qtp651210004-548): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 549 (qtp651210004-549): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 550 (qtp651210004-550): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 551 (qtp651210004-551): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 552 (qtp651210004-552-acceptor-0@161485b2-ServerConnector@76c396f{HTTP/1.1, (http/1.1)}{localhost:33811}): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 553 (qtp651210004-553): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 554 (qtp651210004-554): State: TIMED_WAITING Blocked count: 0 Waited count: 24 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 555 (qtp651210004-555): State: TIMED_WAITING Blocked count: 0 Waited count: 22 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 556 (Session-HouseKeeper-4d1a7915-1): State: TIMED_WAITING Blocked count: 0 Waited count: 2 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 562 (FSEditLogAsync): State: WAITING Blocked count: 6 Waited count: 348 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@19d1f8b6 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:417) app//org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.dequeueEdit(FSEditLogAsync.java:235) app//org.apache.hadoop.hdfs.server.namenode.FSEditLogAsync.run(FSEditLogAsync.java:244) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 564 (Socket Reader #1 for port 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) Thread 565 (IPC Server idle connection scanner for port 39027): State: TIMED_WAITING Blocked count: 1 Waited count: 125 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 569 (org.apache.hadoop.hdfs.server.blockmanagement.PendingReconstructionBlocks$PendingReconstructionMonitor@1d7dee8b): State: TIMED_WAITING Blocked count: 0 Waited count: 5 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.blockmanagement.PendingReconstructionBlocks$PendingReconstructionMonitor.run(PendingReconstructionBlocks.java:259) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 570 (DatanodeAdminMonitor-0): State: TIMED_WAITING Blocked count: 0 Waited count: 413 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 558 (org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor@6315f161): State: TIMED_WAITING Blocked count: 0 Waited count: 248 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor.run(HeartbeatManager.java:541) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 557 (RedundancyMonitor): State: TIMED_WAITING Blocked count: 0 Waited count: 413 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) java.base@11.0.17/java.lang.Thread.sleep(Thread.java:334) java.base@11.0.17/java.util.concurrent.TimeUnit.sleep(TimeUnit.java:446) app//org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$RedundancyMonitor.run(BlockManager.java:4833) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 560 (Block report processor): State: WAITING Blocked count: 0 Waited count: 168 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7119fcd6 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:417) app//org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.processQueue(BlockManager.java:5107) app//org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$BlockReportProcessingThread.run(BlockManager.java:5096) Thread 566 (IPC Server Responder): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) Thread 563 (IPC Server listener on 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) Thread 572 (IPC Server handler 0 on default port 39027): State: TIMED_WAITING Blocked count: 12 Waited count: 1602 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 573 (IPC Server handler 1 on default port 39027): State: TIMED_WAITING Blocked count: 12 Waited count: 1603 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 574 (IPC Server handler 2 on default port 39027): State: TIMED_WAITING Blocked count: 14 Waited count: 1599 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 575 (IPC Server handler 3 on default port 39027): State: TIMED_WAITING Blocked count: 12 Waited count: 1595 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 576 (IPC Server handler 4 on default port 39027): State: TIMED_WAITING Blocked count: 9 Waited count: 1607 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 577 (pool-242-thread-1): State: TIMED_WAITING Blocked count: 3 Waited count: 4 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 579 (org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor@429a6ebe): State: TIMED_WAITING Blocked count: 0 Waited count: 620 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor.run(LeaseManager.java:526) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 580 (org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor@53f681f5): State: TIMED_WAITING Blocked count: 0 Waited count: 248 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor.run(FSNamesystem.java:4093) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 581 (org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeEditLogRoller@6d64b734): State: TIMED_WAITING Blocked count: 0 Waited count: 5 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeEditLogRoller.run(FSNamesystem.java:4135) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 582 (org.apache.hadoop.hdfs.server.namenode.FSNamesystem$LazyPersistFileScrubber@1482f030): State: TIMED_WAITING Blocked count: 0 Waited count: 5 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.server.namenode.FSNamesystem$LazyPersistFileScrubber.run(FSNamesystem.java:4228) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 583 (CacheReplicationMonitor(2132964546)): State: TIMED_WAITING Blocked count: 0 Waited count: 42 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2211) app//org.apache.hadoop.hdfs.server.blockmanagement.CacheReplicationMonitor.run(CacheReplicationMonitor.java:181) Thread 592 (pool-248-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 593 (qtp1531759665-593): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 594 (qtp1531759665-594-acceptor-0@7dfb85d8-ServerConnector@7aafb906{HTTP/1.1, (http/1.1)}{localhost:39221}): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 595 (qtp1531759665-595): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 596 (qtp1531759665-596): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 597 (Session-HouseKeeper-4e3199e3-1): State: TIMED_WAITING Blocked count: 0 Waited count: 2 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 598 (nioEventLoopGroup-8-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:803) app//io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457) app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 599 (org.apache.hadoop.util.JvmPauseMonitor$Monitor@3a0ed71): State: TIMED_WAITING Blocked count: 0 Waited count: 2476 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 601 (Socket Reader #1 for port 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) Thread 602 (IPC Server idle connection scanner for port 35753): State: TIMED_WAITING Blocked count: 1 Waited count: 125 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 606 (Command processor): State: WAITING Blocked count: 0 Waited count: 416 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@cdef922 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.processQueue(BPServiceActor.java:1291) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.run(BPServiceActor.java:1275) Thread 607 (BP-1531809096-172.17.0.3-1697385856100 heartbeating to localhost/127.0.0.1:39027): State: TIMED_WAITING Blocked count: 509 Waited count: 1387 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager.waitTillNextIBR(IncrementalBlockReportManager.java:158) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:719) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:855) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 608 (pool-249-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 591 (org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@44ecbf13): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) java.base@11.0.17/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:114) app//org.apache.hadoop.hdfs.net.TcpPeerServer.accept(TcpPeerServer.java:85) app//org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:145) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 603 (IPC Server Responder): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) Thread 609 (IPC Client (306965244) connection to localhost/127.0.0.1:39027 from jenkins): State: TIMED_WAITING Blocked count: 1347 Waited count: 1348 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1086) app//org.apache.hadoop.ipc.Client$Connection.run(Client.java:1133) Thread 600 (IPC Server listener on 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) Thread 610 (IPC Server handler 0 on default port 35753): State: TIMED_WAITING Blocked count: 0 Waited count: 1243 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 611 (IPC Server handler 1 on default port 35753): State: TIMED_WAITING Blocked count: 0 Waited count: 1242 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 612 (IPC Server handler 2 on default port 35753): State: TIMED_WAITING Blocked count: 0 Waited count: 1242 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 613 (IPC Server handler 3 on default port 35753): State: TIMED_WAITING Blocked count: 0 Waited count: 1242 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 614 (IPC Server handler 4 on default port 35753): State: TIMED_WAITING Blocked count: 0 Waited count: 1242 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 623 (pool-256-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 624 (qtp315182252-624): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 625 (qtp315182252-625-acceptor-0@109265bd-ServerConnector@69e73e0b{HTTP/1.1, (http/1.1)}{localhost:45307}): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 626 (qtp315182252-626): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 627 (qtp315182252-627): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 629 (Session-HouseKeeper-5bb3f6b3-1): State: TIMED_WAITING Blocked count: 0 Waited count: 3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 631 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 632 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 637 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data1/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 5 Waited count: 5 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 638 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data2/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 1 Waited count: 3 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 642 (pool-245-thread-1): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@298da77 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 645 (java.util.concurrent.ThreadPoolExecutor$Worker@652247d8[State = -1, empty queue]): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 646 (nioEventLoopGroup-10-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:803) app//io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457) app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 647 (org.apache.hadoop.util.JvmPauseMonitor$Monitor@6f1e007b): State: TIMED_WAITING Blocked count: 0 Waited count: 2475 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 649 (Socket Reader #1 for port 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) Thread 650 (IPC Server idle connection scanner for port 44717): State: TIMED_WAITING Blocked count: 1 Waited count: 125 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 654 (Command processor): State: WAITING Blocked count: 0 Waited count: 415 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1bf76d01 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.processQueue(BPServiceActor.java:1291) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.run(BPServiceActor.java:1275) Thread 655 (BP-1531809096-172.17.0.3-1697385856100 heartbeating to localhost/127.0.0.1:39027): State: TIMED_WAITING Blocked count: 515 Waited count: 1384 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager.waitTillNextIBR(IncrementalBlockReportManager.java:158) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:719) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:855) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 656 (pool-261-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 622 (org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@57b6801e): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) java.base@11.0.17/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:114) app//org.apache.hadoop.hdfs.net.TcpPeerServer.accept(TcpPeerServer.java:85) app//org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:145) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 651 (IPC Server Responder): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) Thread 648 (IPC Server listener on 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) Thread 657 (IPC Server handler 0 on default port 44717): State: TIMED_WAITING Blocked count: 0 Waited count: 1242 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 658 (IPC Server handler 1 on default port 44717): State: TIMED_WAITING Blocked count: 0 Waited count: 1242 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 659 (IPC Server handler 2 on default port 44717): State: TIMED_WAITING Blocked count: 0 Waited count: 1242 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 660 (IPC Server handler 3 on default port 44717): State: TIMED_WAITING Blocked count: 0 Waited count: 1242 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 661 (IPC Server handler 4 on default port 44717): State: TIMED_WAITING Blocked count: 0 Waited count: 1241 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 670 (pool-268-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 671 (qtp1875028383-671): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) app//org.eclipse.jetty.io.ManagedSelector$$Lambda$229/0x0000000800471840.run(Unknown Source) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 672 (qtp1875028383-672-acceptor-0@49fdfdbd-ServerConnector@692e91fb{HTTP/1.1, (http/1.1)}{localhost:35369}): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:702) app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 673 (qtp1875028383-673): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 674 (qtp1875028383-674): State: TIMED_WAITING Blocked count: 0 Waited count: 21 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 675 (Session-HouseKeeper-19accb11-1): State: TIMED_WAITING Blocked count: 0 Waited count: 2 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 678 (nioEventLoopGroup-12-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:803) app//io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:457) app//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) app//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 679 (org.apache.hadoop.util.JvmPauseMonitor$Monitor@4fd4121a): State: TIMED_WAITING Blocked count: 0 Waited count: 2475 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:192) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 681 (Socket Reader #1 for port 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:1284) app//org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:1263) Thread 682 (IPC Server idle connection scanner for port 35085): State: TIMED_WAITING Blocked count: 1 Waited count: 125 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 686 (Command processor): State: WAITING Blocked count: 0 Waited count: 415 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@50974f58 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.processQueue(BPServiceActor.java:1291) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor$CommandProcessingThread.run(BPServiceActor.java:1275) Thread 688 (BP-1531809096-172.17.0.3-1697385856100 heartbeating to localhost/127.0.0.1:39027): State: TIMED_WAITING Blocked count: 509 Waited count: 1376 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager.waitTillNextIBR(IncrementalBlockReportManager.java:158) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:719) app//org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:855) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 687 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 689 (pool-270-thread-1): State: TIMED_WAITING Blocked count: 0 Waited count: 3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 669 (org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@379268ae): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533) java.base@11.0.17/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285) java.base@11.0.17/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:114) app//org.apache.hadoop.hdfs.net.TcpPeerServer.accept(TcpPeerServer.java:85) app//org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:145) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 683 (IPC Server Responder): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:1520) app//org.apache.hadoop.ipc.Server$Responder.run(Server.java:1503) Thread 690 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 680 (IPC Server listener on 0): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hadoop.ipc.Server$Listener.run(Server.java:1346) Thread 693 (IPC Server handler 0 on default port 35085): State: TIMED_WAITING Blocked count: 0 Waited count: 1241 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 696 (IPC Server handler 1 on default port 35085): State: TIMED_WAITING Blocked count: 0 Waited count: 1241 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 695 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data3/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 1 Waited count: 3 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 697 (IPC Server handler 2 on default port 35085): State: TIMED_WAITING Blocked count: 0 Waited count: 1242 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 698 (IPC Server handler 3 on default port 35085): State: TIMED_WAITING Blocked count: 0 Waited count: 1241 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 699 (IPC Server handler 4 on default port 35085): State: TIMED_WAITING Blocked count: 0 Waited count: 1241 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:458) app//org.apache.hadoop.ipc.CallQueueManager.take(CallQueueManager.java:309) app//org.apache.hadoop.ipc.Server$Handler.run(Server.java:2925) Thread 702 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data4/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 1 Waited count: 3 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 706 (pool-253-thread-1): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@46744811 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 710 (java.util.concurrent.ThreadPoolExecutor$Worker@5054e7fc[State = -1, empty queue]): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 713 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 714 (VolumeScannerThread(/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6)): State: TIMED_WAITING Blocked count: 1 Waited count: 2 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621) Thread 719 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data5/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 2 Waited count: 5 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 720 (refreshUsed-/home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-5443@2/yetus-jdk11-hadoop3-check/src/hbase-server/target/test-data/bf16e1c4-ea3d-b7ac-a2a7-64d7b190db78/cluster_55f3a66b-685b-f451-05e7-3c8d4d8060ec/dfs/data/data6/current/BP-1531809096-172.17.0.3-1697385856100): State: TIMED_WAITING Blocked count: 3 Waited count: 5 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.fs.CachingGetSpaceUsed$RefreshThread.run(CachingGetSpaceUsed.java:179) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 724 (pool-265-thread-1): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@44ff5530 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 728 (java.util.concurrent.ThreadPoolExecutor$Worker@5593a5f4[State = -1, empty queue]): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 729 (FsDatasetAsyncDiskServiceFixer): State: TIMED_WAITING Blocked count: 0 Waited count: 42 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer.run(HBaseTestingUtil.java:596) Thread 731 (LeaseRenewer:jenkins@localhost:39027): State: TIMED_WAITING Blocked count: 41 Waited count: 1318 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:441) app//org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$800(LeaseRenewer.java:77) app//org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:336) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 740 (HMaster-EventLoopGroup-8-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 741 (Listener at localhost/35085-SendThread(127.0.0.1:52753)): State: RUNNABLE Blocked count: 8 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Thread 742 (Listener at localhost/35085-EventThread): State: WAITING Blocked count: 7 Waited count: 22 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1053737 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Thread 743 (zk-event-processor-pool-0): State: WAITING Blocked count: 21 Waited count: 39 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@488db1fb Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 745 (RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@76652b0f Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 746 (RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.Semaphore$NonfairSync@5121d015 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 747 (RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38213): State: WAITING Blocked count: 4 Waited count: 12 Waiting on java.util.concurrent.Semaphore$NonfairSync@7935e989 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 748 (RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5c929d3d Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 749 (RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3f0f8f3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 750 (RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3f0f8f3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 751 (RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@5c0e21e0 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 752 (RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@a21f249 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 753 (RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@4292fd45 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 754 (RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=38213): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@4621b284 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 757 (HMaster-EventLoopGroup-9-1): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 758 (Listener at localhost/35085-SendThread(127.0.0.1:52753)): State: RUNNABLE Blocked count: 10 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Thread 759 (Listener at localhost/35085-EventThread): State: WAITING Blocked count: 4 Waited count: 23 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6b0d9a81 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Thread 760 (zk-event-processor-pool-0): State: WAITING Blocked count: 19 Waited count: 40 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6ebb9e13 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 762 (RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403): State: WAITING Blocked count: 7 Waited count: 70 Waiting on java.util.concurrent.Semaphore$NonfairSync@3b963e54 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 763 (RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=33403): State: WAITING Blocked count: 15 Waited count: 56 Waiting on java.util.concurrent.Semaphore$NonfairSync@70681e30 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 764 (RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33403): State: WAITING Blocked count: 1 Waited count: 35 Waiting on java.util.concurrent.Semaphore$NonfairSync@77b65b2e Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 765 (RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33403): State: WAITING Blocked count: 1 Waited count: 629 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@454f18cd Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 766 (RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=33403): State: WAITING Blocked count: 6 Waited count: 4 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4fa1faae Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 767 (RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33403): State: WAITING Blocked count: 0 Waited count: 4 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4fa1faae Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.ipc.RpcHandler.getCallRunner(RpcHandler.java:68) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 768 (RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=33403): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@7f08f4f4 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 769 (RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=33403): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@3aa2bf10 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 770 (RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=33403): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.Semaphore$NonfairSync@79651f9 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 771 (RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=33403): State: WAITING Blocked count: 0 Waited count: 7 Waiting on java.util.concurrent.Semaphore$NonfairSync@47a2af2f Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345) java.base@11.0.17/java.util.concurrent.Semaphore.acquire(Semaphore.java:318) app//org.apache.hadoop.hbase.ipc.FastPathRpcHandler.getCallRunner(FastPathRpcHandler.java:55) app//org.apache.hadoop.hbase.ipc.RpcHandler.run(RpcHandler.java:82) Thread 774 (MiniHBaseClusterRegionServer-EventLoopGroup-10-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 796 (MiniHBaseClusterRegionServer-EventLoopGroup-11-1): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 818 (MiniHBaseClusterRegionServer-EventLoopGroup-12-1): State: RUNNABLE Blocked count: 9 Waited count: 5 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:312) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:376) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 755 (M:0;8c5d72df62de:38213): State: TIMED_WAITING Blocked count: 9 Waited count: 12340 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) app//org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:623) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 772 (M:1;8c5d72df62de:33403): State: TIMED_WAITING Blocked count: 17 Waited count: 12342 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) app//org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) app//org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:623) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 841 (master/8c5d72df62de:0:becomeActiveMaster): State: TIMED_WAITING Blocked count: 5 Waited count: 19 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hbase.master.ActiveMasterManager.blockUntilBecomingActiveMaster(ActiveMasterManager.java:305) app//org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2486) app//org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:611) app//org.apache.hadoop.hbase.master.HMaster$$Lambda$291/0x0000000800958c40.run(Unknown Source) app//org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) app//org.apache.hadoop.hbase.trace.TraceUtil$$Lambda$276/0x000000080075d440.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 861 (master:store-WAL-Roller): State: TIMED_WAITING Blocked count: 4 Waited count: 12343 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hbase.wal.AbstractWALRoller.run(AbstractWALRoller.java:180) Thread 878 (MiniHBaseClusterRegionServer-EventLoopGroup-12-2): State: RUNNABLE Blocked count: 10 Waited count: 7 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:312) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:376) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 879 (MiniHBaseClusterRegionServer-EventLoopGroup-12-3): State: RUNNABLE Blocked count: 7 Waited count: 4 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:312) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:376) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 886 (AsyncFSWAL-0-hdfs://localhost:39027/user/jenkins/test-data/48e99b63-a68f-156c-32f2-68ddc02602e1/MasterData-prefix:8c5d72df62de,33403,1697385859093): State: WAITING Blocked count: 0 Waited count: 1219 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@49839aec Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 889 (master:store-Flusher): State: TIMED_WAITING Blocked count: 62 Waited count: 108 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2211) app//org.apache.hadoop.hbase.master.region.MasterRegionFlusherAndCompactor.flushLoop(MasterRegionFlusherAndCompactor.java:193) app//org.apache.hadoop.hbase.master.region.MasterRegionFlusherAndCompactor$$Lambda$383/0x0000000800a05840.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 897 (ProcedureDispatcherTimeoutThread): State: TIMED_WAITING Blocked count: 0 Waited count: 111 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) app//org.apache.hadoop.hbase.procedure2.util.DelayedUtil.takeWithoutInterrupt(DelayedUtil.java:81) app//org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher$TimeoutExecutorThread.run(RemoteProcedureDispatcher.java:320) Thread 898 (8c5d72df62de:33403): State: WAITING Blocked count: 4 Waited count: 25 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a323a27 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) app//org.apache.hadoop.hbase.master.assignment.AssignmentManager.waitOnAssignQueue(AssignmentManager.java:2242) app//org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:2264) app//org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$600(AssignmentManager.java:111) app//org.apache.hadoop.hbase.master.assignment.AssignmentManager$1.run(AssignmentManager.java:2204) Thread 899 (normalizer-worker-0): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3fb38977 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) app//org.apache.hadoop.hbase.master.normalizer.RegionNormalizerWorkQueue.take(RegionNormalizerWorkQueue.java:146) app//org.apache.hadoop.hbase.master.normalizer.RegionNormalizerWorker.run(RegionNormalizerWorker.java:191) java.base@11.0.17/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) java.base@11.0.17/java.util.concurrent.FutureTask.run(FutureTask.java:264) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 900 (SnapshotHandlerChoreCleaner): State: TIMED_WAITING Blocked count: 0 Waited count: 124 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 890 (ProcExecTimeout): State: TIMED_WAITING Blocked count: 1 Waited count: 186 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:274) app//org.apache.hadoop.hbase.procedure2.util.DelayedUtil.takeWithoutInterrupt(DelayedUtil.java:81) app//org.apache.hadoop.hbase.procedure2.TimeoutExecutorThread.run(TimeoutExecutorThread.java:56) Thread 891 (WorkerMonitor): State: TIMED_WAITING Blocked count: 0 Waited count: 248 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:279) app//org.apache.hadoop.hbase.procedure2.util.DelayedUtil.takeWithoutInterrupt(DelayedUtil.java:81) app//org.apache.hadoop.hbase.procedure2.TimeoutExecutorThread.run(TimeoutExecutorThread.java:56) Thread 892 (PEWorker-1): State: TIMED_WAITING Blocked count: 84 Waited count: 310 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) Thread 893 (PEWorker-2): State: TIMED_WAITING Blocked count: 48 Waited count: 257 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) Thread 894 (PEWorker-3): State: TIMED_WAITING Blocked count: 54 Waited count: 237 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) Thread 895 (PEWorker-4): State: TIMED_WAITING Blocked count: 72 Waited count: 234 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) Thread 896 (PEWorker-5): State: TIMED_WAITING Blocked count: 33 Waited count: 189 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:165) app//org.apache.hadoop.hbase.procedure2.AbstractProcedureScheduler.poll(AbstractProcedureScheduler.java:147) app//org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1983) Thread 903 (OldWALsCleaner-0): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4e9b90b3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.master.cleaner.LogCleaner.deleteFile(LogCleaner.java:172) app//org.apache.hadoop.hbase.master.cleaner.LogCleaner.lambda$createOldWalsCleaner$1(LogCleaner.java:152) app//org.apache.hadoop.hbase.master.cleaner.LogCleaner$$Lambda$511/0x0000000800ad5040.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 904 (master/8c5d72df62de:0.Chore.1): State: WAITING Blocked count: 33 Waited count: 90 Waiting on org.apache.hadoop.hbase.client.AsyncTableResultScanner@21d0118e Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.Object.wait(Object.java:328) app//org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:156) app//org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:493) app//org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:461) app//org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:422) app//org.apache.hadoop.hbase.MetaTableAccessor.fullScanTables(MetaTableAccessor.java:126) app//org.apache.hadoop.hbase.master.TableStateManager.getTablesInStates(TableStateManager.java:121) app//org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:2194) app//org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:48) app//org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) app//org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) app//org.apache.hadoop.hbase.trace.TraceUtil$$Lambda$276/0x000000080075d440.run(Unknown Source) java.base@11.0.17/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) java.base@11.0.17/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) app//org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 905 (snapshot-hfile-cleaner-cache-refresher): State: TIMED_WAITING Blocked count: 4 Waited count: 13 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 906 (master/8c5d72df62de:0:becomeActiveMaster-HFileCleaner.large.0-1697385860384): State: WAITING Blocked count: 2 Waited count: 5 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3e23859b Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) app//org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:285) app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:254) Thread 907 (master/8c5d72df62de:0:becomeActiveMaster-HFileCleaner.small.0-1697385860393): State: WAITING Blocked count: 2 Waited count: 6 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2caf94e4 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:546) app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:285) app//org.apache.hadoop.hbase.master.cleaner.HFileCleaner$2.run(HFileCleaner.java:269) Thread 922 (HMaster-EventLoopGroup-9-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 923 (HMaster-EventLoopGroup-9-3): State: RUNNABLE Blocked count: 3 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 928 (RegionServerTracker-0): State: WAITING Blocked count: 6 Waited count: 12 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5cb489a8 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 927 (RegionServerTracker-0): State: WAITING Blocked count: 11 Waited count: 24 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@765f54f4 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 954 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 8 Waited count: 17 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2b051542 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 949 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 11 Waited count: 23 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@768cf997 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 957 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 4 Waited count: 9 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2fb61145 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 1005 (region-location-0): State: WAITING Blocked count: 3 Waited count: 6 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3de26df3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1170) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1006 (Registry-endpoints-refresh-end-points): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.hbase.client.RegistryEndpointsRefresher.mainLoop(RegistryEndpointsRefresher.java:92) app//org.apache.hadoop.hbase.client.RegistryEndpointsRefresher$$Lambda$698/0x0000000800bb0840.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1007 (RPCClient-NioEventLoopGroup-7-4): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1008 (RPCClient-NioEventLoopGroup-7-5): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1009 (HMaster-EventLoopGroup-8-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1010 (RPCClient-NioEventLoopGroup-7-6): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1011 (RPCClient-NioEventLoopGroup-7-7): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1034 (RPCClient-NioEventLoopGroup-7-8): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1078 (MiniHBaseClusterRegionServer-EventLoopGroup-4-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1090 (MiniHBaseClusterRegionServer-EventLoopGroup-4-3): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1093 (MiniHBaseClusterRegionServer-EventLoopGroup-5-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1159 (MiniHBaseClusterRegionServer-EventLoopGroup-10-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1175 (MiniHBaseClusterRegionServer-EventLoopGroup-10-3): State: RUNNABLE Blocked count: 3 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1179 (MiniHBaseClusterRegionServer-EventLoopGroup-11-2): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1644 (RegionServer-Restarter): State: TIMED_WAITING Blocked count: 9 Waited count: 1231 Stack: java.base@11.0.17/java.lang.Thread.sleep(Native Method) app//org.apache.hadoop.hbase.replication.TestSyncReplicationStandbyKillRS.waitForRSShutdownToStartAndFinish(TestSyncReplicationStandbyKillRS.java:137) app//org.apache.hadoop.hbase.replication.TestSyncReplicationStandbyKillRS.lambda$testStandbyKillRegionServer$1(TestSyncReplicationStandbyKillRS.java:82) app//org.apache.hadoop.hbase.replication.TestSyncReplicationStandbyKillRS$$Lambda$960/0x0000000800ca7040.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1704 (HMaster-EventLoopGroup-8-3): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1738 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175-SendThread(127.0.0.1:52753)): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Thread 1739 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175-EventThread): State: WAITING Blocked count: 0 Waited count: 2 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7460149d Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Thread 1740 (ReadOnlyZKClient-127.0.0.1:52753@0x365ad98b): State: TIMED_WAITING Blocked count: 2 Waited count: 24 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:328) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$813/0x0000000800c14040.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1743 (zk-event-processor-pool-0): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@615393cd Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1747 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175.replicationSource.wal-reader.8c5d72df62de%2C38435%2C1697385859175.rep,1-8c5d72df62de,45999,1697385859285/8c5d72df62de,38435,1697385859175): State: WAITING Blocked count: 5 Waited count: 9 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@15e3b786 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:341) app//org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.replicationDone(ReplicationSourceWALReader.java:121) app//org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReader.run(ReplicationSourceWALReader.java:149) Thread 1871 (MiniHBaseClusterRegionServer-EventLoopGroup-11-3): State: RUNNABLE Blocked count: 4 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1953 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175-SendThread(127.0.0.1:52753)): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Thread 1954 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,46527,1697385859222/8c5d72df62de,38435,1697385859175-EventThread): State: WAITING Blocked count: 0 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2045228c Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Thread 1955 (ReadOnlyZKClient-127.0.0.1:52753@0x0072515b): State: TIMED_WAITING Blocked count: 2 Waited count: 24 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:328) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$813/0x0000000800c14040.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1956 (zk-event-processor-pool-0): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@53b14619 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1966 (MiniHBaseClusterRegionServer-EventLoopGroup-13-1): State: RUNNABLE Blocked count: 3 Waited count: 2 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1989 (MiniHBaseClusterRegionServer-EventLoopGroup-13-2): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 1997 (regionserver/8c5d72df62de:0.procedureResultReporter): State: WAITING Blocked count: 1 Waited count: 3 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@99f54bd Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.hadoop.hbase.regionserver.RemoteProcedureResultReporter.run(RemoteProcedureResultReporter.java:74) Thread 2041 (MiniHBaseClusterRegionServer-EventLoopGroup-13-3): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) app//org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2156 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222-SendThread(127.0.0.1:52753)): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Thread 2157 (RS_CLAIM_REPLICATION_QUEUE-regionserver/8c5d72df62de:0-0.replicationSource,1-8c5d72df62de,35419,1697385888915/8c5d72df62de,46527,1697385859222-EventThread): State: WAITING Blocked count: 0 Waited count: 1 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@336604e0 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Thread 2158 (ReadOnlyZKClient-127.0.0.1:52753@0x0a7feabd): State: TIMED_WAITING Blocked count: 2 Waited count: 25 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123) java.base@11.0.17/java.util.concurrent.DelayQueue.poll(DelayQueue.java:265) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:328) app//org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$813/0x0000000800c14040.run(Unknown Source) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2159 (zk-event-processor-pool-0): State: WAITING Blocked count: 0 Waited count: 2 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2cdd34ce Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081) java.base@11.0.17/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2166 (Abort regionserver monitor): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: java.management@11.0.17/sun.management.ThreadImpl.getThreadInfo1(Native Method) java.management@11.0.17/sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:197) java.management@11.0.17/sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:154) app//org.apache.hadoop.hbase.util.ReflectionUtils.printThreadInfo(ReflectionUtils.java:175) app//org.apache.hadoop.hbase.util.Threads.printThreadInfo(Threads.java:186) app//org.apache.hadoop.hbase.regionserver.HRegionServer$SystemExitWhenAbortTimeout.run(HRegionServer.java:3564) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:556) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 2216 (master/8c5d72df62de:0.Chore.2): State: WAITING Blocked count: 2 Waited count: 5 Waiting on java.util.concurrent.CompletableFuture$Signaller@61561955 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194) java.base@11.0.17/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1796) java.base@11.0.17/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128) java.base@11.0.17/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1823) java.base@11.0.17/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1998) app//org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:182) app//org.apache.hadoop.hbase.client.AdminOverAsyncAdmin.tableExists(AdminOverAsyncAdmin.java:130) app//org.apache.hadoop.hbase.replication.TableReplicationQueueStorage.hasData(TableReplicationQueueStorage.java:536) app//org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner.preClean(ReplicationLogCleaner.java:81) app//org.apache.hadoop.hbase.master.cleaner.CleanerChore$$Lambda$1116/0x0000000800d55840.accept(Unknown Source) java.base@11.0.17/java.util.ArrayList.forEach(ArrayList.java:1541) app//org.apache.hadoop.hbase.master.cleaner.CleanerChore.preRunCleaner(CleanerChore.java:282) app//org.apache.hadoop.hbase.master.cleaner.CleanerChore.chore(CleanerChore.java:257) app//org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) app//org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) app//org.apache.hadoop.hbase.trace.TraceUtil$$Lambda$276/0x000000080075d440.run(Unknown Source) java.base@11.0.17/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) java.base@11.0.17/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) java.base@11.0.17/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) Thread 2247 (RPCClient-NioEventLoopGroup-7-9): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) app//org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) app//org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) app//org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2248 (Time-limited test): State: TIMED_WAITING Blocked count: 6 Waited count: 3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1798) java.base@11.0.17/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3128) java.base@11.0.17/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1868) java.base@11.0.17/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2021) app//org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:196) app//org.apache.hadoop.hbase.client.Admin.removeReplicationPeer(Admin.java:2007) app//org.apache.hadoop.hbase.replication.SyncReplicationTestBase.shutdown(SyncReplicationTestBase.java:152) app//org.apache.hadoop.hbase.replication.SyncReplicationTestBase.tearDown(SyncReplicationTestBase.java:160) java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) java.base@11.0.17/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) java.base@11.0.17/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.base@11.0.17/java.lang.reflect.Method.invoke(Method.java:566) app//org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) app//org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) app//org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) app//org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) app//org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) app//org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) Thread 2260 (DataXceiver for client DFSClient_NONMAPREDUCE_-1007166148_20 at /127.0.0.1:44644 [Receiving block BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051]): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) app//org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:156) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) java.base@11.0.17/java.io.DataInputStream.read(DataInputStream.java:149) app//org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:210) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:971) app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:908) app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:173) Thread 2262 (DataXceiver for client DFSClient_NONMAPREDUCE_-1007166148_20 at /127.0.0.1:47764 [Receiving block BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051]): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) app//org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:156) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) java.base@11.0.17/java.io.DataInputStream.read(DataInputStream.java:149) app//org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:210) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:971) app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:908) app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:173) Thread 2261 (DataXceiver for client DFSClient_NONMAPREDUCE_-1007166148_20 at /127.0.0.1:53036 [Receiving block BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051]): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: java.base@11.0.17/sun.nio.ch.EPoll.wait(Native Method) java.base@11.0.17/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120) java.base@11.0.17/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124) java.base@11.0.17/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136) app//org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) app//org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:156) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) app//org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.base@11.0.17/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) java.base@11.0.17/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) java.base@11.0.17/java.io.BufferedInputStream.read(BufferedInputStream.java:351) java.base@11.0.17/java.io.DataInputStream.read(DataInputStream.java:149) app//org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:210) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) app//org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:971) app//org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:908) app//org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:173) Thread 2269 (PacketResponder: BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051, type=LAST_IN_PIPELINE): State: WAITING Blocked count: 14 Waited count: 15 Waiting on java.util.LinkedList@215eb79c Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.Object.wait(Object.java:328) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1330) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1402) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2268 (PacketResponder: BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051, type=LAST_IN_PIPELINE): State: WAITING Blocked count: 14 Waited count: 15 Waiting on java.util.LinkedList@70139e53 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.Object.wait(Object.java:328) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1330) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1402) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2267 (PacketResponder: BP-1531809096-172.17.0.3-1697385856100:blk_1073741875_1051, type=LAST_IN_PIPELINE): State: WAITING Blocked count: 14 Waited count: 15 Waiting on java.util.LinkedList@17ee7c41 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.lang.Object.wait(Object.java:328) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1330) app//org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1402) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2303 (process reaper): State: TIMED_WAITING Blocked count: 0 Waited count: 3 Stack: java.base@11.0.17/jdk.internal.misc.Unsafe.park(Native Method) java.base@11.0.17/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234) java.base@11.0.17/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462) java.base@11.0.17/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361) java.base@11.0.17/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1053) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114) java.base@11.0.17/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) java.base@11.0.17/java.lang.Thread.run(Thread.java:829) Thread 2305 (Timer for 'DataNode' metrics system): State: TIMED_WAITING Blocked count: 0 Waited count: 4 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) java.base@11.0.17/java.util.TimerThread.mainLoop(Timer.java:553) java.base@11.0.17/java.util.TimerThread.run(Timer.java:506) Thread 2308 (IPC Client (306965244) connection to localhost/127.0.0.1:39027 from jenkins): State: TIMED_WAITING Blocked count: 0 Waited count: 1 Stack: java.base@11.0.17/java.lang.Object.wait(Native Method) app//org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1086) app//org.apache.hadoop.ipc.Client$Connection.run(Client.java:1133) 2023-10-15T16:24:54,749 ERROR [Abort regionserver monitor {}] server.NIOServerCnxnFactory$1(92): Thread Thread[Abort regionserver monitor,5,FailOnTimeoutGroup] died org.apache.hadoop.hbase.SystemExitRule$SystemExitInTestException: null at org.apache.hadoop.hbase.TestSecurityManager.checkExit(TestSecurityManager.java:32) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT-tests.jar:4.0.0-alpha-1-SNAPSHOT] at java.lang.Runtime.halt(Runtime.java:277) ~[?:?] at org.apache.hadoop.hbase.regionserver.HRegionServer$SystemExitWhenAbortTimeout.run(HRegionServer.java:3565) ~[classes/:?] at java.util.TimerThread.mainLoop(Timer.java:556) ~[?:?] at java.util.TimerThread.run(Timer.java:506) ~[?:?] 2023-10-15T16:24:55,316 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:55,441 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:24:55,441 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:24:55,442 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:24:55,442 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 38, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 571176 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:24:55,446 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:55,446 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:24:55,446 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:24:55,446 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:56,316 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:57,317 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:58,317 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:24:59,317 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:00,317 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:00,329 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:00,330 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:00,330 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:25:00,331 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38341 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:25:00,331 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:00,331 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:25:00,331 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:00,331 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:01,317 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:02,317 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:03,318 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:04,318 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:05,318 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:06,318 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:07,318 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:08,319 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:09,319 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:10,319 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:10,430 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:10,430 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:10,431 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:25:10,432 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48442 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:25:10,432 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:10,432 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:25:10,432 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:10,432 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:11,319 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:12,320 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:13,320 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:14,320 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:14,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:25:14,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:25:15,320 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:15,630 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:15,630 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:15,631 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:25:15,631 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 39, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 591365 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:25:15,632 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:15,632 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:25:15,632 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:15,632 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:16,320 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:17,320 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:18,321 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:18,760 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:25:19,321 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:20,321 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:21,321 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:21,990 ERROR [master/8c5d72df62de:0.Chore.1 {}] normalizer.RegionNormalizerChore(50): Failed to normalize regions. org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:24:21.990Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:22.101Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:22.311Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:22.621Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:23.130Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:24:24.152Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:26.182Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:30.222Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:40.271Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:24:50.312Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:00.331Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:10.432Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:493) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:461) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:422) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.fullScanTables(MetaTableAccessor.java:126) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.master.TableStateManager.getTablesInStates(TableStateManager.java:121) ~[classes/:?] at org.apache.hadoop.hbase.master.HMaster.normalizeRegions(HMaster.java:2194) ~[classes/:?] at org.apache.hadoop.hbase.master.normalizer.RegionNormalizerChore.chore(RegionNormalizerChore.java:48) ~[classes/:?] at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?] at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:25:21,994 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:21,995 DEBUG [master/8c5d72df62de:0.Chore.1 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:21,995 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:25:21,996 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:21,996 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:25:21,996 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:21,996 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:22,100 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:22,100 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:22,101 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:25:22,101 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:22,101 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:22,101 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:22,310 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:22,310 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:22,310 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:25:22,311 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:22,311 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:22,311 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:22,321 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:22,620 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:22,620 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:22,621 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:25:22,621 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:22,621 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:22,621 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:23,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:23,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:23,130 DEBUG [Async-Client-Retry-Timer-pool-0 {}] ipc.AbstractRpcClient(356): Not trying to connect to 8c5d72df62de:35419 this server is in the failed servers list 2023-10-15T16:25:23,131 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:23,131 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:23,131 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:23,322 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:24,140 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:24,140 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:24,141 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:25:24,141 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 6, maxAttempts = 46, timeout = 60000 ms, time elapsed = 2146 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:25:24,142 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:24,142 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:25:24,142 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:24,142 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:24,322 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:25,322 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:26,150 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:26,150 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:26,151 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:25:26,151 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 7, maxAttempts = 46, timeout = 60000 ms, time elapsed = 4156 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:25:26,152 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:26,152 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:25:26,152 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:26,152 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:26,322 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:27,322 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:28,323 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:29,323 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:30,180 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:30,180 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:30,180 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:25:30,181 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 8, maxAttempts = 46, timeout = 60000 ms, time elapsed = 8186 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:25:30,181 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:30,181 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:25:30,181 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:30,182 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:30,323 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:31,323 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:32,323 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:33,324 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:34,324 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:34,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:25:34,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:25:34,924 INFO [PEWorker-5 {}] replication.RemovePeerProcedure(102): There are still 1 pending assign replication queues procedures [63] when removing peer 1, sleep 512 secs 2023-10-15T16:25:34,924 INFO [PEWorker-5 {}] procedure2.TimeoutExecutorThread(82): ADDED pid=68, state=WAITING_TIMEOUT:POST_PEER_MODIFICATION, hasLock=true; org.apache.hadoop.hbase.master.replication.RemovePeerProcedure; timeout=512000, timestamp=1697387646924 2023-10-15T16:25:35,324 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:35,640 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:35,640 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:35,641 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:25:35,641 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 40, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 611376 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:25:35,642 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:35,642 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:25:35,642 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:35,642 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:36,324 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:37,324 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:38,325 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:39,325 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:40,269 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:40,270 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:40,270 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:25:40,271 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 9, maxAttempts = 46, timeout = 60000 ms, time elapsed = 18276 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:25:40,272 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:40,272 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:25:40,272 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:40,272 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:40,325 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:41,325 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:42,325 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:43,326 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:44,326 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:45,326 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:46,326 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:47,326 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:48,326 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:48,760 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:25:49,327 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:50,320 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:50,320 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:50,321 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:25:50,322 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 10, maxAttempts = 46, timeout = 60000 ms, time elapsed = 28327 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:25:50,322 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:50,322 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:25:50,322 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:50,322 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:50,327 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:51,327 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:52,327 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:53,327 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:54,328 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:54,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:25:54,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:25:55,328 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:55,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:25:55,730 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:25:55,731 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:25:55,734 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 41, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 631468 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:25:55,734 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:55,734 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:25:55,734 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:25:55,735 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:56,328 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:57,328 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:58,328 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:25:59,329 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:00,329 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:00,400 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:26:00,400 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:26:00,401 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:26:00,401 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 11, maxAttempts = 46, timeout = 60000 ms, time elapsed = 38406 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:26:00,401 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:00,401 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:26:00,401 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:26:00,402 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:01,329 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:02,329 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:03,329 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:04,329 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:05,330 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:06,330 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:07,330 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:08,330 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:09,331 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:10,331 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:10,480 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:26:10,480 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:26:10,481 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:26:10,481 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for '' in 1588230740 of hbase:meta failed, tries = 12, maxAttempts = 46, timeout = 60000 ms, time elapsed = 48486 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:26:10,481 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:10,482 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:26:10,482 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:26:10,482 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:11,331 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:12,331 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:13,331 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:14,331 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:14,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:26:15,101 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:26:15,332 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:15,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:26:15,750 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:26:15,751 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:26:15,751 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 42, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 651485 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:26:15,751 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:15,751 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:26:15,751 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:26:15,752 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:16,332 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:17,332 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:18,332 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:18,760 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:26:19,332 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:20,333 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:21,333 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:21,600 INFO [ProcExecTimeout {}] assignment.AssignmentManager$DeadServerMetricRegionChore(1502): Found 3 OPEN regions on dead servers and 0 OPEN regions on unknown servers 2023-10-15T16:26:22,001 WARN [master/8c5d72df62de:0.Chore.1 {}] janitor.CatalogJanitor(144): Failed janitorial scan of hbase:meta table org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=13, exceptions: 2023-10-15T16:25:21.996Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:22.101Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:22.311Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:22.621Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:23.131Z, org.apache.hadoop.hbase.ipc.FailedServerException: Call to address=8c5d72df62de:35419 failed on local exception: org.apache.hadoop.hbase.ipc.FailedServerException: This server is in the failed servers list: 8c5d72df62de:35419 2023-10-15T16:25:24.142Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:26.152Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:30.181Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:40.272Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:25:50.322Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:00.402Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:10.482Z, java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at java.lang.Thread.getStackTrace(Thread.java:1602) ~[?:?] at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:144) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:163) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.client.AsyncTableResultScanner.next(AsyncTableResultScanner.java:153) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:493) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:461) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMeta(MetaTableAccessor.java:415) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.MetaTableAccessor.scanMetaForTableRegions(MetaTableAccessor.java:410) ~[classes/:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.scanForReport(CatalogJanitor.java:236) ~[classes/:?] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.scan(CatalogJanitor.java:171) ~[classes/:?] at org.apache.hadoop.hbase.master.janitor.CatalogJanitor.chore(CatalogJanitor.java:136) ~[classes/:?] at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:161) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?] at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) ~[hbase-common-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] at --------Future.get--------(Unknown Source) ~[?:?] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.completeExceptionally(AsyncRpcRetryingCaller.java:109) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.doCall(AsyncSingleRequestRpcRetryingCaller.java:102) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hadoop.hbase.client.AsyncRpcRetryingCaller.lambda$tryScheduleRetry$1(AsyncRpcRetryingCaller.java:139) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:4.0.0-alpha-1-SNAPSHOT] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] ... 1 more 2023-10-15T16:26:22,002 INFO [master/8c5d72df62de:0.Chore.1 {}] hbase.ScheduledChore(145): Chore: 8c5d72df62de,33403,1697385859093-ClusterStatusChore missed its start time 2023-10-15T16:26:22,333 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:23,333 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:24,333 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:25,340 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:26,340 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:27,340 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:28,340 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:29,340 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:30,341 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:31,341 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:32,341 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:33,341 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:34,344 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:34,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:26:35,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:26:35,344 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:35,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:26:35,810 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:26:35,811 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:26:35,811 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 43, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 671545 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:26:35,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:35,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:26:35,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:26:35,812 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:36,344 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:37,344 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:38,344 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:39,345 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:40,345 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:41,345 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:42,345 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:43,345 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:44,346 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:45,346 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:46,346 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:47,346 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:48,346 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:48,761 DEBUG [FsDatasetAsyncDiskServiceFixer {}] hbase.HBaseTestingUtil$FsDatasetAsyncDiskServiceFixer(617): NoSuchFieldException: threadGroup; It might because your Hadoop version > 3.2.3 or 3.3.4, See HBASE-27595 for details. 2023-10-15T16:26:49,347 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:50,347 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:51,347 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:52,347 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:53,347 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:54,348 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:55,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=69 2023-10-15T16:26:55,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=33403 {}] master.MasterRpcServices(1362): Checking to see if procedure is done pid=68 2023-10-15T16:26:55,348 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:55,860 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(545): Start fetching meta region location from registry 2023-10-15T16:26:55,860 DEBUG [Async-Client-Retry-Timer-pool-0 {}] client.ConnectionUtils(553): The fetched meta region location is [region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1] 2023-10-15T16:26:55,861 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.NettyRpcConnection$2(318): Exception encountered while connecting to the server 8c5d72df62de:35419 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] 2023-10-15T16:26:55,862 WARN [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRpcRetryingCaller(177): Call to 8c5d72df62de,35419,1697385888915 for 'hbase:replication' in 1588230740 of hbase:meta failed, tries = 44, maxAttempts = 46, timeout = 1200000 ms, time elapsed = 691596 ms java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:200) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:394) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:93) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:429) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:424) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:117) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:132) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:106) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:202) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$400(NettyRpcConnection.java:82) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.fail(NettyRpcConnection.java:320) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:328) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:303) ~[hbase-client-4.0.0-alpha-1-SNAPSHOT.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[hbase-shaded-netty-4.1.5.jar:?] at java.lang.Thread.run(Thread.java:829) ~[?:?] Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:166) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:131) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) ~[hbase-shaded-netty-4.1.5.jar:?] at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) ~[hbase-shaded-netty-4.1.5.jar:?] ... 7 more 2023-10-15T16:26:55,862 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(64): Try updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 , the old value is region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1, error=java.net.ConnectException: Call to address=8c5d72df62de:35419 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:55,862 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(72): The actual exception when updating region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 is java.net.ConnectException: finishConnect(..) failed: Connection refused 2023-10-15T16:26:55,862 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] client.AsyncRegionLocatorHelper(88): Try removing region=hbase:meta,,1.1588230740, hostname=8c5d72df62de,35419,1697385888915, seqNum=-1 from cache 2023-10-15T16:26:55,862 DEBUG [MiniHBaseClusterRegionServer-EventLoopGroup-12-3 {}] ipc.FailedServers(52): Added failed server with address 8c5d72df62de:35419 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: 8c5d72df62de/172.17.0.3:35419 2023-10-15T16:26:56,348 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:57,348 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:58,348 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:26:59,349 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:27:00,349 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:27:01,349 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:27:02,349 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting 2023-10-15T16:27:03,349 DEBUG [RegionServer-Restarter {}] replication.TestSyncReplicationStandbyKillRS(136): Server 8c5d72df62de,46527,1697385859222 still being processed, waiting